Design Principal for Action on Google

Editor's Notes

• #2: Speaker Intro
• #3: We are all experts at human-to-human conversation. It seems pretty easy, because we’ve been doing it our whole lives. Our whole society is built around it. But conversing only seems easy because it’s familiar, you’ve been doing it since you were born. It has specific rules and conventions. Meaning is negotiated through language, but it’s dependent on context. We humans have gotten pretty good at negotiating meaning from context, without really being aware of it. The key to building a good voice interface is to not fall into the trap of simply converting a Graphical User Interface into a Voice User Interface. This defeats the purpose of using a conversation. People are not going to change how they talk anytime soon, so take what we know about human-to-human conversation to teach our computers to talk to humans. Not the other way around.
• #4: Let’s think about this new concept of a “Voice User Interface”. When a user is navigating a GUI - an user interface, that is graphical, as in web sites or mobile - , they can read text at any location on the web page or application screen. They can skip ahead visually to the part they are interested in, or dismiss as they go. With a VUI, the user is captive to the design … they must listen with (or without) patience to each word before they can hear the one that follows it. So keep this in mind.
• #5: Let’s have have Ido, Developer Advocate for the Assistant explain a bit how to design better Actions
• #6: Number 1. Create a persona. It’s the consistent character, captured by the voice and interactive experience. It’s the face of the experience for the user, who they will be talking to.
• #8: Use case How will a pizza ordering persona be different than a bus schedule one? Beyond the “who” of the brand, what state is user going to be in while they’re using it?
• #9: When do you order a pizza? Hungry, bored, too lazy to make dinner, having a party. Open to having fun When you’re checking your bus schedule, it’s much different Commuters are tired, preoccupied, stressed out. Playful here would probably be annoying. Professional and restrained. Give the info, get out!
• #10: Every little thing in your codebase matters Every word you use is a decision you’re making, and it makes a difference too. For Who your users are, what state they're in, - authentic to the brand or idea you represent
• #12: It’s tempting to draw out a conversation path visually and plug in the dialog, then dive right into the code or start stringing together blocks of individual conversation contexts to create a working action, and then back in to the experience iteratively.
• #13: Don’t do this! Well, you can, but I promise you, it’ll save time and give you a much richer experience to map out the core conversation paths ahead of time. This doesn’t mean just the so-called “happy path”. It doesn’t mean error paths either.
• #14: Instead, write out your core experiences like you would a screenplay. This can be as scrappy as acting it out with a colleague and documenting it on paper, or creating an interactive wizard-of-oz prototype you tweak and play with until you’re ready to start coding. And then, when you draw out your initial vision, keep it at a high level, where the boxes represent entire dialogs or user intents, but leave out the individual wording you’ll use in the interaction.
• #15: Number 3: Remember … in conversations, there are no “errors”. While speech recognition technology isn’t perfect, it’s getting better all the time. However, many problems are not caused by recognition errors, but by other factors …
• #16: There are many reasons an error could happen, from noise, to interruptions, to confusion, and many more. People hardly ever say nonsense. In fact, they’re inherently cooperative. If you ask for them to say something again, they will, but rarely exactly the same way. Don’t repeat the same prompt over and over again, use different variations to help them. Rephrase your question in a different way to get a different possible answer. If you still don’t understand, remember there is a difference between no response and not understanding something that was said. Be conscious about how prompting can slip very easily into condescending territory quickly.
• #17: When a problem happens, imagine what the user hears when your action says “I don’t understand YOU”. This is one of the greatest cause of user frustration and aversions to voice interfaces. People get angry and raise their voice and repeat the same answer again! We need to give people credit for knowing how to speak. Just because they don’t understand a prompt or choices, doesn’t mean they don’t speak the language. So help them be successful. And remember what happens along the way, maintain context.
• #18: In human-to-human conversations, hesitations and corrections happen all the time. Similarly, in human-to-computer interactions, “timeouts” and recognition errors occur. The difference is that humans take cues from each other to get back on track. But with an automated conversation, the corrections need to be pre-planned - not an easy task. The only way to course-correct in advance while still maintaining a natural, comfortable conversation, is to plan for these occurrences as if they were any other turn in a conversation - i.e. to treat them as input that didn’t cause the “error” in and of themselves.
• #19: Remember that users know what they want. Usually errors are not the user misbehaving. Different errors require different strategies. Perhaps the intent is ambiguous to the software trying to interpret it. Because it’s unspoken. But that doesn’t mean the user doesn’t have an intent. And remember, discovery is an intent, poking around is an intent, figuring out the boundaries of a piece of technology is also an intent. But most of the time those are secondary. The user wanted to do something, even if they didn’t say so.
• #20: When writing the sample dialogs, make sure to include different examples for each use case like: “happy path”, “error path”, no input, first time experience vs. returning users, etc. - here the input is not understandable
• #21: Here for example it’s an user, that quits the game Make sure to use conversation language that is matching your personality If your Persona is a young person maybe they will use slang language vs. official formal tone ?
• #22: Give people credit for knowing how to speak and knowing what they want. You don’t have to tell them how to speak the language. And they are cooperative. This means avoiding force-feeding commands such as “to reply to a message, say ‘reply’” or “to hear that again, say ‘repeat’”. People know how to talk. If you throw things at them that way, that’s not a conversation, that’s just another GUI in the guise of a VUI. Those are verbal buttons you’re telling people to press. Don’t try to steer the user back to an original question if they don’t get recognized immediately. There are so many reasons they might not have been. People hardly ever say nonsense. It’s just a misalignment during that meaning construction we talked about. Remember that users know what they want. There’s no such thing as a query with no intent. Perhaps the intent is ambiguous to the engine trying to interpret it. Because it’s unspoken. But that doesn’t mean the user doesn’t have an intent. And remember, discovery is an intent, poking around is an intent, figuring out the boundaries of a piece of technology is also an intent. But most of the time those are secondary. The user wanted to do something, even if they didn’t say so. If you ask for them to say something again, they will, but rarely exactly the same way. Maybe they’ll say it word for word again, but probably louder, clearer, or slightly reworded. Keep this in mind when you write your conversation repair strategies. Don’t create dead ends when something doesn’t work. Give people credit for knowing how to speak and knowing what they want. You don’t have to tell them how to speak the language.
• #23: Environmental - how and where is the user? What are they doing? What type of device are they interacting on? Temporal - how is the experience influenced over time? What happened right before? What’s happening next? How many times have they done this task or conveyed this intent before? Recently? Throughout the course of their relationship with your agent? Situational - where is the user’s frame of mind in relation to what they’re trying to do? Cater to the intent, not a feature. Mental context - what do users know going in? What data can you use going in and in the moment and predictively to enrich the experience? What have users learned from you in the past, are there habits evident in their behavior?
• #24: The conversation should be flowing. It’s your responsibility to offer your users help when things go wrong, so prepare.
• #25: Remember what we’ve said before about the difference of VUI and GUI When a user is navigating a GUI they can skip ahead visually to the part they are interested in, or dismiss as they go. With a VUI, the user can feel like a “prisoner” of the VUI design. They must listen with (or without) patience to each word before they can hear the one that follows it.
• #26: Speaker Intro