Building Conversational Experiences for Google Assistant

Nader Khaled
Solution Developer at Link Development - GDG Alexandria Organizer
Twitter : @naderkhaled92 | Email : nadermousa92@gmail.com

Building Conversational
Experiences

Action
{ conversation api request }
{ conversation api response }
user input
Action response

Ready to
help across
500M+ devices

Devices with the Google
Assistant built-in
These include voice
activated speakers like
Google Home, Android
phones, iPhones etc.
The Google Assistant
A conversation between
you and Google that helps
you get things done in
your world.
Actions on Google
How developers can
extend the assistant

JapaneseGerman
FR French
CA French
KoreanUK English
AU English
CA English
US English
Italian
Spanish BR Portuguese
LATAM Spanish
Hindi & IN English
A growing audience:
languages
Swedish
Danish
Norwegian Indonesian
Russian
Thai
Dutch

Hey Google, talk
to __________.
Hey Google, connect
me with __________.
Hey Google,
get __________.

Ok Google, talk
to Personal Chef
Sure, here’s
Personal Chef
What are you
in the mood
for?
What protein
would you like
to use?
Speech to Text
NLP
Knowledge Graph
ML Ranking
User Profile
Text to Speech
Speech to Text
Text to Speech
...
Invoke Personal
Chef action
Parse query and
generate
response
Well, it’s kind of
cold outside, so
I’d like...

Well, it’s kind of cold outside, so I'd like
something to warm me up, like a hot
soup, and I want it fast.

“
Intent Matching — Match and categorize user
utterances to an intent.
Entity Extraction — Identify key words and
phrases spoken by the user.@

Ok Google, talk
to Personal
Chef
Sure, here’s
Personal Chef
What are you
in the mood
for?
What protein
would you like
to use?
Speech to Text
NLP
Knowledge Graph
ML Ranking
User Profile
Speech to Text
Text to Speech
...
...
...
Trigger
WELCOME event
...
NLP:
Intent Matching
Entity Extraction
Well, it’s kind of
cold outside, so
I’d like...

Entities:
homemade
cannoli
“Find me a recipe
for homemade
cannoli”
User Says
Response Webhook
“Find me a recipe
for homemade
cannoli”
“Here’s the best
one I can find”

AUDIO_OUTPUT SCREEN_OUTPUT
Support different surface capabilities
developers.google.com/actions/assistant/surface-capabilities

Support speech and display text
developers.google.com/actions/assistant/responses
conv.ask(new SimpleRespone({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number, like 42. What do you have in mind?',
text: 'Howdy! I can tell you fun facts about ' +
'almost any number. What do you have in mind?'
}));

Display basic cards
developers.google.com/actions/assistant/responses
conv.ask('Math and prime numbers it is!')
conv.ask(new BasicCard({
text: '42 is an even composite number. It is composed of three distinct prime ' +
'numbers multiplied together. It has a total of eight divisors. 42 is an ' +
'abundant number, because the sum of its proper divisors 54 is greater ' +
'than itself. To count from 1 to 42 would take you about twenty-one…',
title: 'Math & prime numbers',
buttons: new Button({
title: 'Read more',
url: 'https://example.com',
}),
image: new Image({
url: 'https://example.google.com/42.png',
alt: 'Image alternate text',
}),
}));

Account Linking
https://developers.google.com/actions/identity/
Multi platform
Google Sign In + OAuth linking
Assistant only
Google Sign In only
Google Sign In for Assistant configurable in two modes

Sample Actions
developers.google.com/actions/samples/actions

Smart Home Device Integration
developers.google.com/actions/smarthome/
Dim the lights in the living
room a little bit.
What lights are on in the
living room?
Make it warmer.
Turn off all the lights.
What is the temperature
inside?
How many lights are on?

Ok Google, ask Personal Chef for a hot soup recipe
Trigger Phrase Action Name
Developer Specified
Action
Preposition
Action Phrase
Developer Specified
Working Examples: Let’s speak to Domino’s
Ask Dr. Doggy if dogs can eat chocolate
Ok Google, let’s talk to Personal Chef
Trigger Phrase Action Name
Developer Specified
In Dialogue Discovery: Explicit Triggering

In Dialogue Discovery: Implicit Triggering
Hey Google, I
want to work out.
Hey Google, I want
to play a game.
Hey Google, what’s
the surf report?
Hey Google, tell
me a joke.

GUI : VUI
What’s the difference?

A Persona is conveyed through:
• Tone
• Word and phrase choices
• Functional design
• Style
• Technique
• Voice
And it is based on:
• Your user population
• Their needs
• The imagery &
qualities associated
with your brand

Cowabunga, dude!
What flavors do you crave?
The bus is approaching your stop now.
Your trip will take 45 minutes.

Buy a pizza
Getting there soon
Purchase a ticket
Arriving soon
Look at your order Review your schedule

[2] Think outside the box.
Literally!

code code code code code
code code code code
code code “ dialog string” code
code code code
code code code code code
code “dialog string” code code
code code code code code code code
code code code
code code
code
code code
code
code
code code code
code “dialog string” code code code
code code
code code code
code code code code code code code code
code code code code code code code code
code
STRUCTURE CODE

[3] In a conversation,
there are no “errors”

Give users credit
They know how to talk
Avoid force feeding commands.
(such as “to reply to a message,
say ‘reply’” or “to hear that
again, say ‘repeat’”.)
People hardly ever say
nonsense. They just weren’t
parsed correctly. There was a
misalignment in meaning.
They know what they want
There’s no such thing as a query with
no intent. The user wanted to do
something, even if they didn’t say so.
They want to help
Don’t create dead ends when
something doesn’t work.
If you ask for them to say
something again, they will, but
rarely exactly the same way.
Honor that they’re invested in
outcome of a repair attempt.

Use the context
Environmental
Where is the user?
What are they doing?
What type of device
are they using?
Situational
What’s their intent?
Where is the user’s
frame of mind?
Temporal
What happened right before?
What’s happening next?
How is the experience
influenced over time?
Behavioral
What do users know
going in?
What data can you use to
enrich the experience
predictively or to cut out
unnecessary steps?

Codelab: Build Actions (Level 1)

codelabs.developers.google.com → Assistant

Create either a specific project ID or share
your individual project ID via Google Forms to
have your Action recognized as part of the event
series:
1. Share your Project ID via
bit.ly/buildactions-projects
OR
1. Go to the console:
bit.ly/buildactions-console
2. Start a new project beginning with BAFYC
followed by the planned Action name
Start creating your Action - get recognized

Console
bit.ly/buildactions-console
Build with Templates
developers.google.com/actions/templates
Build with Dialogflow
developers.google.com/actions/dialogflow
Build with Actions SDK
developers.google.com/actions/sdk
Resources to get started and support
Documentation
developers.google.com/actions
Codelabs
codelabs.developers.google.com
Write and Deploy Functions
bit.ly/firebase-get-started
Deploy your fulfillment webhook using
Cloud Functions for Firebase
bit.ly/aog-cloud-functions

Wait… there’s more
1. Check out the recording of one webinar with the Assistant team for all
bit.ly/buildactions-webinarvideo
2. Your Actions listed on the Google Assistant event series page:
developers.google.com/events/buildactions
3. Be part of the Actions on Google Community with publishing you Action
developers.google.com/actions/community (and get a shirt + cloud credits for a year)

Actions on Google Community Program
developers.google.com/actions/community
You are automatically invited to the program with publishing
your first Action and will receive a
Cool T Shirt + $200 Cloud Credits per month for a year
The Assistant team also recognizes milestones:
Traction Engagements Globalization

Have a great evening!
#GoogleAssistant #ActionsOnGoogle #AOGDevs #DialogFlow

Building Conversational Experiences for Google Assistant

More Related Content

Similar to Building Conversational Experiences for Google Assistant (20)

Recently uploaded (20)

Building Conversational Experiences for Google Assistant