Chatbots and Digital Assistant Analysis
1 like430 views
Digital assistants like Siri and chatbots on messaging platforms could help brands interact directly with customers at scale. However, usability tests found that people expect assistants to have high conversational abilities and include the brand in responses. Behavioral analyses showed most assistant queries are brief, fact-based questions rather than open conversations. Facebook brand page analysis found many customer posts go unanswered and interactions can include complex conversation chains. A chatbot prototype had success with structured surveys but struggled with open-ended conversations that people often abandoned quickly. While chatbots offer opportunities, the complexity of managing open-ended conversations means early brand bots will likely focus on structured, succinct exchanges.
Chatbots and Digital Assistant Analysis
- 1. David Wright David.Wright@kantar.com Digital Assistants – a brand’s best friend?
- 2. What are Digital Assistants? Personal Assistants Customer Service Assistants Chatbots
- 3. Messenger “Chatbot” Global Reach Messengers are where that conversation is at. WeChat, China - 700 million users Facebook Messenger – 1 billion users KIK – 300 million users Line, Japan – 215 million users
- 4. “A brand’s best friend?” - The experiments Usability Test How do people interact with a DA? Behavioural Analysis SIRI + Facebook Messenger interaction patterns Facebook Brand Page Analysis What do direct brand to consumer conversations look like now? Messenger ‘Chatbot’ Prototype and Analysis 1 2 3 4
- 5. 1 Location based questions “If I ask for a baked chicken recipe, or how to remove a stain, why not link straight to a 'how to' video on YouTube from a reputable brand?” “I need to adjust the way I talk, so Google understands me". Weather and Time Questions Information Searches “I like it when Siri replies to me (via voice), not when it gives me a list of tiny links" “I like it when there is a photo in the result” Usability Test
- 6. Usability test summary People set the conversational bar very high The knowledge base is controlled by Apple or Google. The brand is not included in the conversation. These assistants are not a brand’s best friend. For brands though… “This was the first time I used SIRI in two years. Wow it’s got really good. When I first used it didn’t understand me. I didn’t use it again." 1
- 7. Behavioural data analysis Word frequency for mobile searches that contain “How” Are device makers now ceding the customer relationship to search engines? TV FIX MONEY WORK REMOVE CAR ANDROID PLAY PHONE FREE HAIR IPHONE COOK RESET APP XBOX CARD WATER CLEAN UNLOCK GALAXY CALORIES BABY DOG HOME INSTALL ONLINE BECOME GOOGLE FACEBOOK OPEN WORTH BOX GROW MUSIC OIL CONNECT MOVIES PAY WINDOWS “How do I connect YouTube from Phone to TV?” “How do I edit my contacts on my Samsung Galaxy?” 2
- 8. Behavioural data analysis What to people do on SIRI? 70K SIRI SEARCHES 4K IMAGES TAGGED 2 Movies 18% Music 1% Nature 6% Other 9% Sports Faces 48% Sports Logos 18% Wolfram 8% Maps 8% Bing Search 63%
- 9. Behavioural data analysis “Do Messenger Patterns indicate how people might want to interact?” 52% 18% 30-60 seconds 5%60-90 seconds 30-60 seconds 0-30 seconds Individual messenger sessions are brief and succinct. 2
- 10. Behavioural test summary Brands need to be part of the digital conversation in their domain space Are succinct as more than half of individual messenger interactions are less than 30 seconds. Are often a Q & A hunt for information. Images can bring this information to life. Digital conversations: 2
- 11. Facebook brand page analysis “What do brand consumer digital interactions look like now?” u = user (the person who first posted to the page and started a conversation chain) b = the brand 0 = other users have decided to comment within the conversation chain. 3
- 12. Facebook Brand Page Analysis 26% of customer posts were not answered 22% were responded to and the conversation was complete 3
- 13. A large and complex long tail of interactions. For the 13K page posts there were 942 different chain types. For 10% of multi-chain conversations the brand was not present. Facebook brand page analysis 3
- 14. Facebook brand page analysis Tesco had the smallest % of unanswered posts and the highest chain variation. 3 0 10 20 30 40 50 60 0 100 200 300 400 500 600 700 800 900 1000 Boots Lidle Aldi UK Tesco Marks and Spencer % Unanswered Chain Types
- 15. Facebook page test summary Public Facebook posts are likely the tip of the iceberg. Open conversation ‘chains’ can be complex. Facebook brand page conversations are public. In KLM’s case 7 times more messages across the private channel. Brands are likely to prefer direct/private digital communication. For brands though… 3
- 16. Meet “Kit” the chatbot Structured Survey Open “AI” chat 4 Tested with 150 Kantar Employees
- 17. Messenger “Chatbot” prototype and analysis Finally we can participate! “ 4
- 18. The good A simple “structured” survey with elements including “buttons” and “quick replies” worked! Say “Hi” to Kit! 4
- 19. The bad - aka attempts at open conversations 0 10000 20000 30000 40000 50000 60000 70000 80000 1 3 5 7 9 1113151719212325272931333537394143454749515355575961636567 Total Duration of Chat (seconds) Majority of interactions 1-15 Minutes. Conversation ID Seconds Conversations that last 16 hours Conversations that span 1-2 hours Although individual interactions are short and sweet a user pause and pick up the conversation at any time. It was easy to get lost in open chat and forget the purpose of the interaction. 52% of posts to the bot in “open conversation mode” were one word only. 4
- 20. The bad - aka too smarty pants too quickly ”What do you do to treat it?” “I have a broken ankle”. “I am wearing a boot.” “Has it been a tough few days?” “Yes.” “Ok thanks. Please remember to say Hi again!” “Do you not feel sorry for me?”” 4
- 21. 4 AI tools – initially subtle and in the background
- 22. A brand’s best friend? Key conclusions Chatbots combined with vast Messenger networks offer an unprecedented opportunity for brands to connect with customers at scale. However chatbot creation is not straight forward. Customers expect brief and accurate interactions. However direct brand to consumer conversations can be complicated and nuanced. Early adopters and influencers will try to stump an AI enabled chatbot which may create a PR risk. DAs/chatbots have the potential to become a brand’s best friend. In the short term, due to the complexities of an AI managing open conversation, it is likely the brand bot experiences will start simply and rely heavily on buttons and other structured UI elements.