How to build and publish a google home app with dialogflow

Moses Sam Paul August 10, 2018
Chapter 6: How to build a Google Home App with
Dialogflow |Overview
medium.com/swlh/chapter-6-how-to-build-a-google-home-app-with-dialogflow-overview-4549d92d8d6a
Series Content:
Chapter 6: Google Home App Tutorial: Overview
Chapter 7: Google Home App Tutorial: Conversation Design
Chapter 8: Google Home App Tutorial: Environment Setup
Chapter 9: Google Home App Tutorial: Fulfillment via Inline Editor
Chapter 10: Google Home App Tutorial: Fulfillment via Cloud Datastore
Chapter 11: Google Home App Tutorial: Fulfillment via Webhook
Chapter 12: Google Home App Tutorial: App Deployment
In this series, we are going to see how to write and deploy a cloud function on firebase and
use the google cloud’s datastore as the backend and the action.google / google assistant
as the front end. Don’t panic, we are gonna break down each “term” below.
Let’s try to grasp the big picture here.
Remember this high-level flow? [If not, I strongly suggest that you finish theother 5
chapters]
1/5

From this chapter onwards, we are gonna develop another new app just for google home
using dialog flow and here, the fulfillment section is going to use a cloud function on
firebase and pull data from the google cloud’s datastore to fulfill the intent. And the front-
end or the user interaction is going to happen through google home so the revised version
incorporating all the technology pieces is given below.
New Terms Alert!
wakeup!!!!
1. Firebase: Firebase is a mobile and web application development platform developed
2/5

by Firebase, Inc. in 2011, then acquired by Google in 2014.
Explore > https://console.firebase.google.com/
2. Cloud Function: Cloud Functions for Firebase lets you automatically run backend
code in response to events triggered by Firebase features and HTTPS requests. Your
code is stored in Google’s cloud and runs in a managed environment. There’s no
need to manage and scale your own servers.
Read more > https://firebase.google.com/docs/functions/
3. Google Cloud Platform: Google Cloud Platform, offered by Google, is a suite of
cloud computing services that runs on the same infrastructure that Google uses
internally for its end-user products, such as Google Search and YouTube.
Read more > https://console.cloud.google.com/
4. Google Cloud Datastore: Google Cloud Datastore is a NoSQL document database
built for automatic scaling, high performance, and ease of application development.
Read more > https://cloud.google.com/datastore/docs/concepts/overview
5. Actions for Google: Actions on Google is a program for developers of “Actions”
(software applications) that run via Google Assistant on devices such as the Google
Home smart speaker and the Google Pixel smartphone and on the Google Allo
mobile app. Actions for Google includes toolkits, tutorials, and other community
resources.
Developers can build two types of Actions.
Direct Actions are simple; according to ‘The Verge’: “ask for information, get an
answer. Ask to turn off the lights, the lights turn off. Ask to play a song, and it plays.
Conversation Actions, in contrast, are more back and forth. “ Conversational
Actions use tools from Dialogflow, a company that Google bought and whose tools
are integrated into Actions for Google.
Read More: https://console.actions.google.com/
Conversation Steps:
3/5

Lets walk through the conversation steps with an example.
#1: User say “Give me a motivational Quote”
#2 Google Home sends the statement to Google Assistant
#3 Google assistant uses action for google to transcribe the speech to text
#4 Dialog Flow receives the text and figures out which agent to send this to
#5 Dialogflow’s agent identifies the intention of the user and passes to the right intent
#6 Dialogflow’s intent uses entities to store parameter values [Quote_Type: Motivational ]
*Intents are optional
#7 Dialogflow’s intent passes the request along with entities to fulfillment
#8 Fulfilment uses webhook to call the cloud function
A webhook is an API concept that's growing in popularity. A webhook (also called a
web callback or HTTP push API) is a way for an app to provide other applications
with real-time information. A webhook delivers data to other applications as it
happens, meaning you get data immediately.
#9 Cloud function executes the business logic and connects to the data store and pulls
relevant data.
#10 Fulfillment delivers the response (JSON object) from the webhook
#11 Dialogflow returns the response to Google Assistant
4/5

#12 Actions for Google built for google assistant synthesizes text to audio to google home
#13 Google Home responds to user with a motivational quote.
Steps involved in building a Google Home App:
1. Design the conversation
2. Setup the environment
3. Build with Dialogflow
4. Test
5. Deploy
6. Measure
Now that the big picture is clear, in the next chapter, we’ll see how to design the
conversation.
5/5

DialogFlow |Conversation Design
medium.com/swlh/chapter-7-how-to-build-a-google-home-app-with-dialogflow-conversation-design-63d6a1402ed0
In the previous chapter, we have set the context of the app that we are trying to build for
google home and the conversation flow of the same at a high level.
In this chapter, we are gonna see how to design a conversation that will eventually become
our app.
Series Content:
Conversation design is a design language based on human conversation (similar to how
material design is a design language based on pen and paper).
1/5

The more an interface leverages human conversation, the less users have to be taught
how to use it.
Conversation Design is a synthesis of several design disciplines such as,
Voice User Interface design (VUI)
Interaction Design
Visual Design
Motion Design
Audio Design
UX writing.
“A man dressed in black standing under a wave-like white concrete structure” byUrfan
Hasanov on Unsplash
The role of a conversation designer is like that of an architect, mapping out what users
can do in a space, while considering both the user’s needs and the technological
constraints.
Design Tips:
1. Create a Persona
2. Think outside the box, literally (beyond the screens ) :P
3. Remember in conversations, there are no errors (everything has to be handled)
Design Process:
2/5

How to write a Simple Dialogue.
1. Focus on one user persona and one key use case.
2. Find a partner and role-play the conversation, with one person pretending they’re the
user and the other pretending they’re the system persona. Record the conversation. If
you don’t have a partner, you’ll have to switch between playing both roles.
3. Transcribe the conversation. This is the first draft of your sample dialog.
4. Step through the dialog, saying the user’s lines and playing each of the system
persona’s lines in the text-to-speech (TTS) in which it will be rendered. If the TTS
doesn’t sound good, rewrite it or use Speech Synthesis Markup Language (SSML) to
change its performance.
5. Repeat steps 1–4 with different user personas and key use cases.
Sample Draft of Dr. Motivation
Simple enough? In the next chapter, let’s try and implement a small chat bot that pulls
motivational quotes from the cloud data store and fulfills through the cloud function for
firebase.
Don’t worry if you don’t understand what a cloud datastore is or what a cloud function on
firebase is. We’ll slowly unravel each term and build up form scratch. Now that all the
theory part is over, trust me, It’s gonna get interesting!
4/5

DialogFlow | Environment Setup
medium.com/swlh/chapter-8-how-to-build-a-google-home-app-with-dialogflow-environment-setup-3547993e99a4
Now that we understand the high level flow of dialogs and designed a sample conversation.
Let’s go ahead and set up the environment to start building Dr. Motivation.
Series Content:
We’ll cover the marked part in this chapter.
1/11

Step 1: Create a project on console.actions.google.com
Step 2: Skip the category
Step 3: Set invocation
2/11

Step 4: Click on Actions in the left column & Add your first Action
3/11

Step 5: Clicking build will open up the dialogflow console with intent already pre-filled, just
click on create
Step 6: Two intents are available already..the default fallback intent and the default
welcome intent
4/11

Default welcome intent handles all the pleasantry inputs by the user
default fallback intent handles all the inputs that aren’t matched to any intents we have
created/ configured
Step 7: Create a new intent (LookingForQuote) and add a hard-coded text response to it
5/11

Step 8: Test out if the intent is triggered
6/11

cool…so when the user say “give me a quote” the intent “LookingForQuote” is triggered
and the default response is pushed out.
Step 9: Create Entity.
Let’s say you want to give different types of quotes to the user. Type could be friendship,
romantic, motivational etc.
7/11

Step 10: Go back and edit the “LookingForQuote” intent to accommodate the “QuoteType”.
Step 11: Set Parameters and make it “required” and define the prompts so that it’s easy for
the user to give the right response.
8/11

Step 12: Save the intent and test out the agent.
9/11

if the user asks for a quote our agent now
prompts the user to be specific w.r.t the entities
we have created.
You can notice that irrespective of the type of
quote we are always getting the same response
because we have just one “hard-coded”
response.
Let’s make it a little more dynamic by enabling
the fulfillment module in the next chapter.
10/11

DialogFlow | Fulfillment via Inline Editor
medium.com/swlh/chapter-9-how-to-build-a-google-home-app-with-dialogflow-fulfillment-via-inline-editor-4b6cc21ba14a
Now that we have setup the environment, created necessary agent, intents and entity and
tested them. Let’s move on to the fulfillment module where we’ll use the inline editor to
create a relevant response based on the entity (QuoteType)
Series Content:
Recap and which part of the big picture we are tackling now.
1/16

Step 1: Enable Fulfillment for the LookingForQuote intent.
Step 2: — Fulfillment via inline editor [Powered by Cloud functions for Firebase]
2/16

Remove everything within the inline editor. Let’s try to write from scratch one line at a time.
Nothing to be afraid of, stay with me.
Here goes. The first line.
'use strict';
Strict:
The purpose of “use strict” is to indicate that the code should be executed in “strict mode”.
With strict mode, you can not, for example, use undeclared variables.
All modern browsers support “use strict” except Internet Explorer(IE) 9 and lower: Who
uses IE anyway….unless…
3/16

ok moving on.
const functions = require('firebase-
functions');
Const basically defines something that
this isn’t gonna change within the
program forever. We will be importing
firebase’s functions module here.
Then a few more consts each of which will
hold our intents. These strings should
match the name of the intents.
'use strict';
const functions = require('firebase-
functions');
const {dialogflow} = require
('actions-on-google');
const WELCOME_INTENT = 'Default
Welcome Intent';
const FALLBACK_INTENT = 'Default
Fallback Intent';
const LOOKING_FOR_QUOTE_INTENT =
'LookingForQuote';
…
Now we are gonna store the dialogflow
object within a const called app.
const app = dialogflow();
Next, we are gonna define the intents and
instantiate a “conv” object that will use the
ask function to talk to the user.
app.intent(WELCOME_INTENT, (conv) => {
conv.ask("welcome to Dr.Motivation! Ask for a quote about friendship or romance
or motivation");
});
Remember to enable webhook under the fulfillment section within the “default welcome
intent”.
similarly we are going to define the other two intents as well..don’t worry about this being
simple, as our goal is to make this fulfillment run without any errors first.
one for fallback intent
app.intent(FALLBACK_INTENT, (conv) => {
conv.ask("Stop mumbling & speak up");
});
4/16

one for our custom intent
app.intent(LOOKING_FOR_QUOTE_INTENT, (conv) => {
conv.ask("Friends are therapists, we can drink with");
});
finally we are gonna return the dialogflow object app when the fulfillment is called
exports.dialogflowFirebaseFulfillment = functions.https.onRequest(app);
the full code looks like this
'use strict';
const functions = require('firebase-functions');
const {dialogflow} = require ('actions-on-google');
const WELCOME_INTENT = 'Default Welcome Intent';
const FALLBACK_INTENT = 'Default Fallback Intent';
const LOOKING_FOR_QUOTE_INTENT = 'LookingForQuote';
or motivation");
});
});
});
Step 3: Deploy the function
Step 4: Test the function
5/16

check out the Fulfillment response by clicking the “Diagnostic Info” ..you will see the quote
we have mentioned within the code now.
Let’s try the default fallback without enabling fulfillment via webhook and with it.
Without Webhook Enabled :
7/16

The diagnostic info basically has just one RAW API response
8/16

Now, let’s go ahead and enable fulfillment via webhook for the fallback intent..if you haven’t
done so already.
output:
9/16

now that webhook call is enabled we are getting a fulfillment response as well.
Step 5: Let’s add more functionality to the code.
Since we are getting QuoteType as a parameter, let’s try and use that within our code via
simple If else statements.
first define the QuoteType entity
const QUOTE_TYPE_ENTITY = 'QuoteType';
then extract the parameter value, store it in a variable…match it to values and push
different quotes each time.
const quote_type = conv.parameters(QUOTE_TYPE_ENTITY).toLowerCase();
if (quote_type == "motivational") {
conv.ask("Don't be a lazy ass");
} else if (quote_type == "frienship") {
} else if (quote_type == "romantic") {
conv.ask("something romantic");
} else {
conv.ask("get off your ass and work instead of talking to me");
}
});
Test it out:
11/16

You can also check out the firebase console for error logs.
12/16

go to your fire base console..or under the inline editor click on the link
ok..so conv.parameters is not a function but an object array so change () to []
conv.parameters(QUOTE_TYPE_ENTITY)
to
conv.parameters[QUOTE_TYPE_ENTITY]
Final Code is given below:
'use strict';
13/16

or motivation");
});
});
const quote_type = conv.parameters[QUOTE_TYPE_ENTITY].toLowerCase();
conv.ask("Don't be a lazy ass");
} else if (quote_type == "frienship") {
conv.ask("something romantic");
} else {
conv.ask("get off your ass and work, instead of talking to me");
}
});
Step 6: Deploy & Test:
14/16

Obviously, this can’t be the final version, if you gotta make it really dynamic you gotta
connect it to a database to pull data based on the user input.
In the next chapter, we are gonna use Google’s cloud datastore which is a type of database
to pull data from them and serve quotes to our users.
16/16

DialogFlow | Fulfillment via Cloud Datastore
medium.com/swlh/chapter-10-how-to-build-a-google-home-app-with-dialogflow-fulfillment-via-cloud-datastore-
706d6067294b
In this chapter we are gonna see how to connect google cloud’s datastore to have a set of
quotes with different categories and pull them based on the user inputs.
Series Content:
Recap and which part of the big picture we are tackling now:
1/10

Let’s understand what datastore is and how is it different first.
What’s Google Cloud Datastore ?
Google Cloud Datastore is a NoSQL document database built for automatic scaling, high
performance, and ease of application development.
What’s a NoSQL database?
A NoSQL (originally referring to “non SQL” or “non relational”) database provides a
mechanism for storage and retrieval of data that is modeled in means other than the tabular
relations used in relational databases.
Comparison with traditional databases
While the Cloud Datastore interface has many of the same features as traditional
databases, as a NoSQL database it differs from them in the way it describes relationships
between data objects. Here’s a high-level comparison of Cloud Datastore and relational
database concepts:
2/10

Unlike rows in a relational database table, Cloud Datastore entities of the same kind can
have different properties, and different entities can have properties with the same name but
different value types. These unique characteristics imply a different way of designing and
managing data to take advantage of the ability to scale automatically.
Enough with the theory, let’s begin.
Let’s setup our datastore now
Step 1: Login to your google cloud console:
Step 2: Select your project and Click on Datastore.
3/10

Step 3: Create Entity
Remember entity is just an item (row). Kind (QuoteTable) is the table name. Leave the
name space as default.
4/10

Quote, QuoteID & QuoteType are the properties [columns] . Make sure that the QuoteType
and QuoteId are indexed as we’ll be using these properties for filtering based on the user
input.
Now that a basic data store is setup in the next chapter we’ll see how to pull data from this
from our cloud function.
Let’s go back to our inline editor.
Till now we have edited only the index.js file, but this cloud datastore is a dependecy that
has to be added to the package.json file.
5/10

"@google-cloud/datastore": "1.1.0"
Back to index.js file.
Am gonna make the code as simple as possible, don’t expect coding standards :)
1. instantiate the object
const Datastore = require('@google-cloud/datastore');
// Instantiate a datastore client
const datastore = Datastore();
2. Define the queries
const query1 = datastore.createQuery('QuoteTable').filter('QuoteType', '=',
'Motivational');
'Friendship');
"Romantic");
3. Run the query, capture the return JSON file and print a specific Quote.
6/10

return datastore.runQuery(query1).then(results => {
conv.ask(results[0][1].Quote);
});
} else if (quote_type == "friendship") {
});
});
} else {
}
});
Let us take one piece of this and understand.
if (quote_type == "friendship") {
});
The results is a JSON object that has two parts. The first part hold the filtered data and the
second part holds some info. If we print these values and see the firebase log, we’ll find the
below.
so we take the first object and within that am hard-coding the second quote to be shown all
the time, this is shitty programming but you can always loop and show a random quote.
4. Final code within index.js
// See https://github.com/dialogflow/dialogflow-fulfillment-nodejs
// for Dialogflow fulfillment library docs, samples, and to report issues
'use strict';
7/10

or motivation");
});
});
'Motivational');
'Friendship');
"Romantic");
});
} else if (quote_type == "friendship") {
});
});
} else {
}
});
5. Deploy & Test
8/10

Cool, so what did we do here?
Instead of hard-coding quotes within the code, we are now pulling quotes from the google
cloud data store using the Fulfillment’s inline editor.
Well, this isn’t enough, is it ? When we are writing complex functions we might need better
debugging and better control over the code and the packages we are leveraging. So it’s
better to set up a development environment and start tinkering our code over there
In the next chapter, we’ll see how to use the firebase command line interface and a JS
editor to move the development to our local machine and deploy it as a cloud function and
use webhook to do the same.
10/10

DialogFlow | Fulfillment via Webhook
medium.com/heptagon/chapter-11-how-to-build-a-google-home-app-with-dialogflow-fulfillment-via-webhook-
7117b51dfc00
In this chapter, we are going to learn how to get Fulfillment done via a webhook.
Series Content
As you can see below, the big picture has changed a bit because now we know that
Fulfillment can be done via in-line editor or via webhook. In this chatper we are gonna be
seeing how to use the webhook and use a cloud function to pull data from the google data
store.
1/14

Building Fulfillment with Webhook:
Step 1: Enable Webhook
Step 2: Install Node and NPM
2/14

Download | Node.js
Download the Node.js source code or a pre-built installer for your platform, and start
developing today.nodejs.org
NPM is node package manager. npm is distributed with Node.js- which means that when
you download Node.js, you automatically get npm installed on your computer.
npm
npm is a separate project from Node.js, and tends to update more frequently. As a result,
even if you’ve just…www.npmjs.com
Open your terminal/ command prompt and check if node and npm are available with these
commands
node -v
and for npm
npm -v
Step 2: Install Firebase package
npm install -g firebase-tools
-g means the node package is being installed globally.
You can read more about this package here.
3. Login to Firebase via browser by running the following command
firebase login
4. Find your present working directory
pwd
5. Make a few folder
mkdir DrMotivation
6. Get into the new folder
cd DrMotivation/
3/14

7. Initiate Firebase here. The tool gives you an option to install dependencies with npm.
firebase init functions
So locally /users/sam/DrMotivation is the foder where the firebase functions will be stored.
The default firebase directory for this directory is the name of your “Agent / google cloud
project name” …
Select javascript as the language
ESLint to catch portable bugs — yes [no idea what this means]
A linter or lint refers to tools that analyze source code to flag programming errors, bugs,
stylistic errors, and suspicious constructs.
ESLint is a tool for identifying and reporting on patterns found in ECMAScript/JavaScript
code, with the goal of making code more consistent and avoiding bugs. In many ways, it is
similar to JSLint and JSHint with a few exceptions: https://eslint.org/docs/user-guide/getting-
started
Install Dependencies — yes
4/14

The output would like this.
Firebase Initialization Complete!
8. Let’s see what’s inside the project folder / our present working directory (pwd)
open .
Cool, remember the inline editor page within dialogflow had package.json and index.js..we
have the same files here but with a littel but more. The node modules folder has all the
packages that are required to run this locally.
9. Open the functions folder within your favorite text editor ( I usesublime)
5/14

Open Index.js
Cool, now that the entire working environment is setup, let’s move on to adding code to the
index.js file and publish it as a cloud function.
10. Copy the index.js code from our dialogflow’s inline editor and the required
dependencies into the package.JSON
11. Deploy the firebase function
firebase deploy
12. Errors!
6/14

replaced == with === let’s try and run now.
OK, 6 errors has come down to 3…so the then.results expects a return so…
have made few changes to the code. (return true :P)
7/14

Now, let’s try running this.
Nice, those errors are gone, but we have “cannot find module ‘actions-on-google’.
Try tunning
cd functions
npm install
That should fix it.
Now, lets come out of the functions folder
cd ..
pwd
Make sure you are in the project folder.
Deploy firebase functions again
firebase deploy --only functions
Voila!
8/14

Let’s go to dialogflow’s fulfillment page and enable webook.
ok, let’s test!
9/14

Looks like it’s working!
10/14

Just to be sure, am gonna add 1 more item to romantic category and pull that quote item
current data store:
Adding one more romantic quote.
Go back and edit the index.js file . I know it’s shitty programming but bear with me, we are
building up!
return true;
Now we should get the new quote we just added to the datastore
Deploy the firebase function and test it out on dialogflow. Worked!
11/14

worked! Let me put the package.JSOn and
inde.JS files below
package.JSON
{
"name": "functions",
"description": "Cloud Functions for
Firebase",
"scripts": {
"lint": "eslint .",
"serve": "firebase serve --only
functions",
"shell": "firebase functions:shell",
"start": "npm run shell",
"deploy": "firebase deploy --only
functions",
"logs": "firebase functions:log"
},
"dependencies": {
"@google-cloud/datastore": "1.1.0",
"actions-on-google": "2.0.0-alpha.4",
"dialogflow": "^4.0.3",
"dialogflow-fulfillment": "0.3.0-
beta.3",
"firebase-admin": "~5.13.0",
"firebase-functions": "^2.0.0"
},
"devDependencies": {
"eslint": "^4.12.0",
"eslint-plugin-promise": "^3.6.0"
},
"private": true
}
Index.JS
'use strict';
or motivation");
});
13/14

});
'Motivational');
'Friendship');
"Romantic");
if (quote_type === "motivational") {
//getQuote(query1)
return true;
});
} else if (quote_type === "friendship") {
return true;
});
} else if (quote_type === "romantic") {
return true;
});
} else {
}
});
Summary: What did we learn in this chapter?
We have setup a local development environment to work on the cloud functions for firebase
and made sure it works.
In the next chapter, we are gonna see how to deploy this on Google Home app store, so
that anyone can access it.
14/14

DialogFlow | App Deployment
medium.com/heptagon/chapter-12-how-to-build-a-google-home-app-with-dialogflow-app-deployment-9596bd74d9ad
Alright, now that our basic chatbot is up and running let’s integrate with Action on Google
so that we can access it via Google Assistant.
Series Content
Step 1:
1/4

In the popup that opens, make sure your intents are there and click on manage assistant
app.
That should open the Actions on Google Console..with the test simulator..test out the app
here. Now you can find the actual responses.
2/4

Cool, so the actions on dialog flow is running perfectly on actions on google console. Let’s
move to deploy.
Deploy:
Fill the necessary details. Privacy policy is important so try to generate on fromhere
A draft of mine is below:
Location Targeting: Select the countries where you want this to be released
Surface Capabilities: No Changes here.
Release:
3/4

Complete the release checklist. Indicate you have completed the required steps. Then,
click the Submit for Review button.
Select testers. In the next dialog, click on the button to whitelist testers to try your alpha or
beta version. Then, enter the email addresses for the testers. You can specify up to 20
testers for alpha testing and up to 200 testers for beta testing. Testers must have a Google
account (@gmail.com) or a G Suite account. You cannot whitelist multiple users via a
Google group email alias. Click Save to complete your submission.
Invite testers. You should send an invitation email to whitelisted testers with a link to the
alpha or beta version of your Action.
Leave your mail id below, if you wanna test the app :)
This finishes our second series on building an app for Google Home. Hope you guys found
it useful, if yes, clap as long as long as you can hold that mouse button.
4/4

How to build and publish a google home app with dialogflow

More Related Content

Similar to How to build and publish a google home app with dialogflow (20)

More from Moses Sam Paul Johnraj (11)

Recently uploaded (20)

How to build and publish a google home app with dialogflow