Watson DevCon 2016 - Engage and Be Engaging: Building Compassionate and Personalized Conversational Systems
7 likes1,859 views
The document discusses the development of compassionate and personalized conversational systems by IBM, highlighting the need for understanding user preferences, emotions, and interactions to enhance engagements. It presents various scenarios demonstrating how cognitive assistants can adapt their responses based on individual user circumstances, thereby improving the user experience. Core frameworks and models necessary for achieving these personalized interactions include understanding people deeply, refining human-computer interaction, and implementing effective communication strategies.
Watson DevCon 2016 - Engage and Be Engaging: Building Compassionate and Personalized Conversational Systems
- 1. Engage and be Engaging: Building Compassionate and Personalized Conversational Systems Rama Akkiraju IBM Distinguished Engineer
- 2. Please note IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBM’s sole discretion. Information regarding potential future products is intended to outline our general productdirection and it should not be relied on in making a purchasing decision. The information mentionedregarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential futureproducts may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and theworkload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 11/21/16World of Watson 2016 2
- 3. Consider this conversation Presentation Title 8/11/16 Watson Developer Conference 2016 3 What kind of cuisine do you prefer? What’s a good place to eat in San Francisco? Are you a vegetarian? Italian, or Mexican, or Indian, or Chinese Do you like a fine dining experience or fast food? Yes Are you driving, taking a taxi or using public transportation? Fine dining Do you prefer a restaurant that has a parking lot of its own, or are you okay to find street parking? Driving I can recommend three choices: Greens on Marina, Gracious Madre on Mission, and Enjoy Chinese on Kearney… Parking lot would be good Works! But takes too many turns to arrive at a solution and solution is not personalized
- 4. Consider this revised conversation Presentation Title 8/11/16 Watson Developer Conference 2016 4 What’s a good place to eat in San Francisco? Better! Takes far fewer turns and offers much more personalized experience What kind of cuisine do you prefer? Joan, because you are a vegetarian, open to experiences, prefer fine dining, and are likely to drive given you are currently renting a car, I’d recommend Greens, a fusion vegetarian Restaurant on 2 Marina Blvd. It is a 4.1/5-rated vegetarian restaurant with ample parking and is within 1 mile from your current location, so you could even walk to it. Italian, or Mexican, or Indian, or Chinese Cognitive assistant is demonstrating understanding of Joan’s preferences, personality, life events and personalizing answers accordingly.
- 5. Consider this conversation Presentation Title 8/11/16 Watson Developer Conference 2016 5 There are proven, effective ways to deal with bullying and bullies. Would you like me to help you? I’m getting bullied in school. I’m so upset. Are you facing verbal, physical or social bullying? Yes. Please. Where has this been happening? I guess, social and verbal bullying. I can’t focus on my studies. My situation is so hopeless. Why do you think they are bullying you? On the school bus, during recess, after school, during projects – everywhere Have you tried avoiding those who are bullying you? I think it’s because I’m overweight. They call me names. It’s so demeaning. Here is a short video on how to deal with bullying. This can help you. I can’t. they are my teammates. I have to do a project with them. Works! But lacks compassion
- 6. Consider this revised conversation Presentation Title 8/11/16 Watson Developer Conference 2016 6 Kathy, I’m so sorry to hear that you are facing bullying in school. Don’t feel distressed. You are not alone in this fight. There are proven, effective ways to deal with bullying and bullies. I’m here to listen and to help you in any way I can. Would you like me to help you? I’m getting bullied in school. I’m so upset. Are you facing verbal, physical or social bullying? Yes. Please. Oh! That is indeed unfortunate! Don’t lose hope, Kathy. You can manage the situation. I’m here to help you. First, let me ask you this. Where has the bullying been happening? I guess, social and verbal bullying. I can’t focus on my studies. My situation is so hopeless. Better! Compassionate companion
- 7. Consider this revised conversation Presentation Title 8/11/16 Watson Developer Conference 2016 7 I see. Why do you think they are bullying you? On the school bus, during recess, after school, during projects – everywhere I’m sorry to hear that. That is indeed very insensitive of them to say that. Have you tried avoiding those who are bullying you? I think it’s because I’m overweight. They call me names. It’s so demeaning. I see. Kathy, I can help you in the following ways. I have access to some videos that may help you. After you had a chance to watch them and grasp the techniques suggested, if you wish, we can also do some role play. That way, you can be prepared to deal with them next time you face them. I can’t. They are my teammates. I have to do a project with them. Cognitive assistant is demonstrating understanding of Kathy’s emotional state and is working with her in a compassionate manner. Better! Compassionate companion
- 8. To build compassionate and personalized conversational systems, three core models are needed: Presentation Title 8/11/16 Watson Developer Conference 2016 8 Naturally3 Interact2 People1 Understand people at a deeper level Understand and respond in various mediums in which interactions can occur Interact2 naturally3 with people1 Input types: Text, speech, gestures Mediums: Computers, mobile devices, robots, avatars Understand styles of human interaction and optimize human-computer interaction
- 9. Building block 1: People modeling/ user modeling Naturally3 Interact2 People1
- 10. People are complex beings with many facets “Who are these people?” “How do they make decisions?” “What do they value?” “How do they think?” “What is their context?” “What are their preferences?” “What are their emotions?” “How do they act?” “How are they feeling?” “What are their skills?” “How do they express themselves?”
- 11. User modeling: Our framework An individual takes action based on the combination of his/her unique being and environment Act Be Feel Context Think Options Explore & Decide Inner State Environment Outer State
- 12. User modeling: Our framework Act Search Preferences Communications Decisions Commitments Purchases Context Life Style, Events Sociological Economic Political Technological Options Price Promotions Products/Services Place Feel Perceptions Emotions Sensations Attitudes Influences Sentiments Be Personality Needs, Values Beliefs Motives Identity Goals, Ambitions Interests Think Knowledge Skills Opinions Cognitive Style Explore & Decide Choices Consequences Session Intent Time
- 13. 11/21/16 Watson Developer Conference 201613 Ready to use Still in R&D
- 14. Use Personality Insights to engage with individuals at personalized level 11/21/16 Watson Developer Conference 201614 Watson-pi-demo.mybluemix.net Source: https://www.army.mil/article/78562/Leaving_the_battlefield__Soldier_shares_story_of_PTSD
- 15. Emotional Analysis helps build empathetic systems 11/21/16 Watson Developer Conference 201615 sentiment-and-emotion.mybluemix.net
- 16. Use Tone Analyzer to understand and fine tune your message 11/21/16 Watson Developer Conference 201616 tone-analyzer-demo.mybluemix.net
- 17. Building block 2: Human interaction patterns Naturally3 Interact2 People1
- 18. Natural interactions among people 11/21/16 Watson Developer Conference 201618 Verbal (expressive, aggressive, passive) , Non-verbal (gestures, facial expressions, postures)
- 19. Dialog strategies Start Giving an extra Acknowledging Need Description of Need Anger Acknowledging w/o encouraging Refocus statements Active Listening Possibility of mistake Admitting mistake Allowing venting Apology Smiles Arranging Follow-up Need cannot be fulfilled on the spot Assurance of effort Assurance of result Mistake has been made Bonus buyoff Broken record Uncooperative customer Closing positively Common Courtesy Completing Follow-up Contact Security Aggressiveness Disengaging Distraction Frustration Empathy statement Expediting Expert Recommend ation Explain Reasoning or action Embarrassment Face-Saving Out Conflict Finding Agreement Points Following up Helpless Offering Choice Empowering Preventive strike Privacy insurance Privacy concern Probing question Pros and Cons Providing Alternatives Providing Takeaway Confusion Providing Explanation Questioning instead of stating Referral to supervisor Referral to 3rd party Lost focus Refocus Inappropriate behavior Setting Limits Critical Neutral mode Summarize the conversation Silence Thank-you Timeout Use customer name Verbal Softeners When questionYou’re right Action Negative Emotion Monologue End External Giving Emotions General States Gratitude Statement Happiness Work by Michal Shmueli-Scheuer, Jonathan Herzig, Guy Feigenblat, David Konopnicki
- 20. Tone-aware Expressions in Conversations 11/21/16 Watson Developer Conference 201620 food-coach. mybluemix.net
- 21. 11/21/16World of Watson 2016 Page 21
- 22. Auto-generated response in customer care domain 11/21/16 Watson Developer Conference 201622 Agent responses are auto- generated.
- 23. Bot personas and expressive paraphrasing 11/21/16 Watson Developer Conference 201623
- 24. Personalized conversational systems 11/21/16 Watson Developer Conference 201624 Based on customer’s emotion and personality the cognitive bot personalizes responses Detect user emotion Bot understands user’s personality and preferences and provides recommendations Change bot expression based on user emotion
- 25. Building block 3: Understand various mediums in which human-computer interaction can occur Naturally3 Interact2 People1
- 26. Tone-aware Expressions in Conversations 11/21/16 Watson Developer Conference 201626 Speech-to-Text Text-to-Speech speech-to-text- demo.mybluemix.net text-to-speech- demo.mybluemix.net
- 27. Deriving human expressions from speech, images and video Speech – Non-verbal clues: pauses, volume, intonation, pitch 11/21/16 Watson Developer Conference 201627 Video, images – Gestures – Facial expressions – Eye contact – Posture – Tone of voice – Distance
- 28. Bringing it all together: Building compassionate and personalized conversational systems Naturally3 Interact2 People1
- 29. Deep-dive: Watson Food-Coach App Demonstrates how to incorporate tone and emotion detection into Conversations and to design compassionate dialog flow 29
- 30. 11/21/16World of Watson 2016 Page 30 Intents Entities
- 31. Food Coach Dialog Flow 31
- 32. Building Compassionate and Personalized Conversational Agents 11/21/16 Watson Developer Conference 201632 Naturally3 Interact2 People1 • We are developing the platform and the building blocks for implementing compassionate and personalized conversational agents. • Checkout our services, Build your conversational apps and give us your feedback!
- 33. Notices and disclaimers continued Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third- party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml. 33 11/21/16World of Watson 2016