Introduction to Voice Design
1 like629 views
Jeff LeBlanc, a UX director and software developer, presents on voice design, emphasizing its benefits and challenges in user interaction. Key topics include automated speech recognition, natural language understanding, and the importance of designing conversations that feel natural. The presentation also discusses error handling, persona creation, and the advantages of combining voice with visual interfaces.
Introduction to Voice Design
- 1. Introduction to Voice Design Presented by Jeff LeBlanc, Director of User Experience Boston UX 1
- 2. About the Presenter Jeff LeBlanc • Director of UX at Boston UX and ICS • Software developer for 20+ years • Adjunct faculty at WPI teaching HCI Contact me at jeffl@bostonux.com 2
- 3. About Boston UX • Providing inspired UX design for embedded devices • Design arm of Integrated Computer Solutions • Over 30 years of software excellence • Specialists in intuitive interface design for touch- and voice-powered smart devices 3
- 4. Fascinated by the Future • Genius • Billionaire • Playboy • Philanthropist • Futurist 4
- 5. Turning Science Fiction Into Fact • Fiction has inspired innovation and design for years • Jules Verne inspired Simon Lake • H.G. Welles inspired Robert Goddard 5
- 6. Turning Science Fiction Into Fact 6
- 7. Tony Stark’s Mansion Good morning. It’s 7:00 a.m. The weather in Malibu is 72 degrees with scattered clouds. The surf conditions are fair with waist-to-shoulder high lines. High tide will be at 10:52 a.m. 7
- 8. Voice Interaction • Voice is an excellent input modality • Allows for hands-free interaction and multitasking • Fast, often beats typing or manual controls • Nuances of voice output can add to experience • Devices can be designed: • Screen first, add voice later • Voice first, add screen later 8
- 9. Voice Interaction – Why not? • May not be suitable for public spaces, privacy concerns • Medical and HIPAA situations • Disturbing others in your living space • Can add to cognitive load • Remember instead of recognize 9
- 10. Design Considerations • Automated Speech Recognition • What did I say? • Natural Language Understanding • Do what I mean! • Voice User Interface (VUI) 10
- 11. Automated Speech Recognition • DRAGON was started in 1975 • Dragon Naturally Speaking - 1997, recognized around 100 words • Major platforms reporting 95% recognition rate • Same as human to human accuracy • Automated Speech Recognition • Accuracy achieved by cloud processing • Local processing, less accurate 11
- 12. Automated Speech Recognition • Intel Realsense initiative, started in 2012 • Free SDK, limited hardware (only supported Creative camera) • Accuracy level has varied between versions of the SDK and target hardware • In practice, simple commands recognized with 40% to 60% confidence 12
- 13. Automated Speech Recognition • Intel Realsense initiative, started in 2012 • Free SDK, limited hardware (only supported Creative camera) • Accuracy level has varied between versions of the SDK and target hardware • In practice, simple commands recognized with 40% to 60% confidence 13
- 14. Natural Language Understanding • The harder problem, requires lots of computing and “intelligence” • Neural nets, machine learning, etc. • Challenge is getting meaning from a set of words • What I say: “Do you have a pen?” • What I mean: “I need a pen and if you have one please give it to me now.” 14
- 15. Natural Language Understanding “Skip the spinning rims! We’re on the clock!” 15
- 16. Natural Language Understanding • 1950 - Turing test • Test of a machine’s ability to communicate as a human • 1990 - Loebner prize • Artificial intelligence competition • 2017- Alexa prize • Build “a socialbot that can converse coherently and engagingly with humans on popular topics for 20 minutes.” 16
- 17. Natural User Interfaces • Designing a UI that has a “natural” feel • Need to be able to hold a conversation and understand context • Conversation is more than a single “turn” • “Jarvis, turn on kitchen lights.” • “Yes sir.” Vs • “Jarvis, turn on the lights.” • “Which lights, sir?” • “The kitchen lights.” • Rules of conversation are learned by being human 17
- 18. VUI Design “Designing Voice User Interfaces: Principles of Conversational Experiences” Cathy Pearl, 1st edition 18
- 19. VUI Design • Confidence thresholds and confirmation • “Jarvis, order me some paper towels” • > 80% “Yes sir, ordering you more paper towels” • Implicit confirmation - repeating part of the question in the answer • 45% - 79% “Sir, I think you said to order more paper towels, is that correct?” • Uncertain, ask for confirmation • < 45% “I’m sorry sir, I didn’t understand what you said” • Non-speech confirmation • “Jarvis, turn on the lights” 19
- 20. VUI Design • Command and Control - typically “one off” voice commands • Triggers • Push to talk • The “open mic” problem • Wake words • Alexa et al parse for wake words locally, then send sound to cloud 20
- 21. VUI Design • How do we handle error conditions? • No speech detected • Speech detected but not recognized • Recognized but system does the wrong thing • Escalation error handling • prompt for more info • don’t blame the user! 21
- 22. VUI Design • Natural conversations use variety of words for same meaning • Design for intents and support synonyms of meaning • Jarvis: “Shall I render using proposed specifications?” • Tony Stark: “Thrill me.” • Saying “yes” - yup, yep, yeah, uh huh, please do, go for it, etc. 22
- 23. VUI Design • Intent - an action that the user wants to take • Intent can sometimes be inferred by object • “Show my calendar” versus “Add a meeting to my calendar” • Designs VUIs to handle intents first 23
- 24. VUI Design • Limitation of voice is serial output • With visual, we can parallel process • Support the ability to interrupt output (barge in) • Jarvis: “Sir, at 19% power, the odds of reaching that altitude ...” • Tony Stark: “I KNOW THE MATH! DO IT!!” • Break up long lists into smaller chunks • “Would you like to hear more?” 24
- 25. VUI Design • Consider multiple modalities • Combinations of screen and voice are very powerful • Amazon Echo Show • Example: a voice query that displays results on a touch screen 25
- 26. VUI Design • Handling error condition of speech detected but not recognized • Solution: provide a list of probable matches, scored and sorted by context or other knowledge • N-Best lists solution • Example: “find” versus “fine” • Make your error messages as useful as possible • Talk as a person, not as software 26
- 27. VUI Design • Disambiguation - speech is understood but we don’t have enough information to take action • “I’d like a large, please” • Often requires a conversation • Not enough information - “What is the weather in Springfield?” • Too much information - “I have a cough and a fever” • Properly handling negation - “I’m not feeling well” - “Great to hear!” 27
- 28. VUI Design • Support for conversations is challenging • Requires memory of “past” parts of conversation • Support for pronouns • Jarvis: “The render is complete.” • Tony Stark: “Hey, I like it. Fabricate it.” • Building a “dialog” in the original sense of the word 28
- 29. VUI Design • Speech output vastly influences the overall experience • Stephen Hawking and DECtalk - Perfect Paul • JARVIS - Paul Bettany doing voice overs • Pre-recorded voice • Limited responses, but more emotional • Computer generated voice • More flexible, less nuanced 29
- 30. VUI Design • You are designing a persona, not a device • Siri, Jarvis, Friday, Alexa, Cortana: they all have personalities • You are not designing a person, but a persona • You can say “I” or “me”, but avoid “we” so as to not group with humans • Design your persona as you would in any design activity, with traits and quirks • Jarvis – British butler, said “Sir” • Friday – Irish assistant gal, said “Boss” 30
- 31. VUI Design Consider human rules of conversation to make the interactions as natural as possible • Use transitional markers • “Next, I’ll need to know…” • Be helpful and respectful within the persona • Provide acknowledgements and feedback • “Okay, got it.” • Avoid jargon and tech speak • Don’t: “Sorry, there was a server error” • Do: “Hm, I’m having trouble doing that right now” 31
- 32. VUI Design • When working with pure voice, you are always working with a blank slate • No affordances, nothing to recognize • Provide prompts • “You can do things like…” • Consider your intents and synonyms carefully • Test, test, test! 32
- 33. Thanks for Attending Contact me @ jeffl@bostonux.com 33