The Software Challenges of Building Smart Chatbots - ICSE'21
Download as PPTX, PDF0 likes500 views
The document discusses the complexities and challenges of developing effective chatbots, including the integration of natural language processing (NLP) and the need for specific technologies in their construction. It highlights the increasing importance of chatbots in various sectors, predicting significant growth in their usage and market value by 2024. Additionally, the document outlines Xatkit's approach to chatbot development, emphasizing its orchestration platform and design language to simplify the process for developers and non-technical users alike.
The Software Challenges of Building Smart Chatbots - ICSE'21
- 1. The Software Challenges of Building Smart Chatbots – ICSE’21 TB @xatkit – xatkit.com
- 2. Context
- 3. Jordi Cabot - @JordiCabot Gwendal Daniel - g__daniel
- 4. Can you help us create a chatbot to help newcomers to write good bug reports on GitHub? Sure! It’s just a set of questions & answers, this will be pretty simple! Narrator It wasn’t. Once upon a time in 2018 (we were young and naïve)
- 5. We were asked to build a bot. We ended up building a bot platform But at least we’ve learnt some things along the way that today we want to share with you 3 times so far!
- 6. The plan for today • THEORY (30’) • HANDS-ON PRACTICE (1H45’) • You can watch • Better if you watch and try (installing included!) • Ask questions on the chat at anytime, we’ll keep an eye on them
- 7. Why you all need to have some bot building notions
- 9. And more predictions: - Business Insider experts predict that by 2022, banks can automate up to 90% of their customer interaction using chatbots - By 2024, consumer retail spend via chatbots worldwide will reach $142 billion—up from just $2.8 billion in 2019. Chatbot market is worth over 1 Billion USD
- 11. Chatbots <-> Software engineers • Software Engineers -> Chatbot – Chatbots are a new type of software artefact that we’ll need to consider in our dev projects (Multi-experience dev) – Plenty of research opportunities to uncover • Chatbot -> Software Engineering – Bots are more and more playing a supportive role in many SE tasks (e.g. in GH) so better to know how they work and their limitations
- 12. Basic concepts
- 13. What is exactly a Chatbot? • Bot are software components • Chatbot = Bot + Chat interface • Voicebot = Bot + Voice Interface Voicebot Chatbot Bot
- 14. VoiceBot • Alexa, Google Assistant ,… • They hide a chatbot inside 1. Speech-to-text 2. Process the text as a chatbot 3. Text-to-speech to reply
- 15. Chatbot vs LiveChat • LiveChat: a person is behind • They can be combined – Bot Person: when the bot doesn’t know how to answer (or the visitor is specially “interesting”) – Person Bot: when no support person is available or the client is not worth
- 16. Bots and NLP/NLU • At the core, a NLU/NLP engine with two key missions – Classify the user utterance (i.e. input text) in one of the intents (set of possible questions/requests) defined in the bot – Identify the parameters (called « entities ») from the utterance
- 17. We ignore open-domain bots (e.g. LaMDA) • They can chat about everything but – A complete black-box – ``Useless’’ for a specific business – You could use them (or QA systems) as a fallback
- 18. Why chatbot development is so challenging?
- 19. Chatbots combine all the complexity of bots with the complexity of NL (including typos, trolls, sentiment analysis…)
- 20. Many pieces to assemble Input/Output Messaging channels External Platforms NLU Engine Events!
- 21. Chatbots are complex software systems Conversation Logic Text Processing External Services Messaging Platforms Deployment Evolution Maintenance Tests
- 22. Jordi Cabot¹, Loli Burgueño², Robert Clarisó², Gwendal Daniel², Jorge Perianez-Pascual³,and Roberto Rodriguez-Echeverria³ ¹ICREA, Barcelona, Spain ²Open University of Catalonia, Barcelona, Spain ³University of Extremadura, Cáceres, Spain Testing challenges for NLP-intensive bots BotSE ‘21
- 23. NLP for chatbots in 5 min
- 24. NLP in a chatbot implies Parameters (entities) I would like to order a medium pepperoni pizza Intent Input text (utterance) • Understand what the user is trying to ask the bot • Identify and gather the data in the request that the bot needs to construct the response
- 25. NLU approaches • Regular expressions • Require an almost exact match. Not used • Grammar-based approaches • Useful for small domains with a very specific vocabulary and constructs • E.g. Stanford Parser • Neural Networks - Multiclass classifiers 1. You provide some training sentences for each intent (the "classes" ). 2. The classifier learns from these sentences 3. It assigns any new input text to one of the classes (with %confidence)
- 26. Example: bot that greets you, informs about the schedule and lets you buy a pizza Greetings Hi Good morning Good afternoon Hello Morning! I’d like to buy a pizza I want a pizza Can I order a pizza? Buy Are you open? Is it open now? What are your opening hours? At what time do you start? Schedule If a user says, “I would like to order a pizza” what is the closest intent? Not that easy when intents start to overlap (e.g. offering a job vs looking for a job: “can I offer you a job?” vs “do you have any job offer”)
- 27. Entity recognition • We need to understand the intent but also identify the entities • Much easier if PizzaType is an enumeration ({ Pepperoni, Veggie, Hawaian ,…}) instead of the any string ChooseYourPizza I’d like a PizzaType pizza I’m taking a PizzaType pizza I think I’m going for a PizzaType
- 28. Common tricks in NLU Engines • Tokenizers split the sentences into words • Stemmers identify the lemma of the word (e.g. to focus on the verb, not on the tense of the verb for matching purposes) • Synonyms and paraphrasing are an automatic way to expand the training sentences • Entity augmentation also generalize training sentences • E.g. What’s the weather in Barcelona -> What’s the weather in @city (the intent will match no matter what city the user asks for, no need to train the classifier with all the cities of the world as it will already know the list of cities itself). • Augmentation is possible for a large number of data types and common concepts (measures, countries, dates,…) https://medium.com/@jseijas/how-to-build-your-own-nlp-for-chatbots-9b4c08eb03a9
- 29. Beyond intent classification • Great bots go beyond providing the right answer, e.g. they could show some empathy • Several language models (for toxicity detection, for sentiment analysis, language identification, emoji support…) can be easily integrated in bots
- 30. PRO tip: Monitor your bots • You will fail to predict: • What users want to know • How they express their intentions • Careful with self-learning • Filter out what the bot needs to add as training sentences to avoid strange derivations
- 31. What bot building tool should I use?
- 32. >100 bot platforms • Rasa, BotPress, Chatfuel, Inbenta, Botsify, Flow XO, Xatkit… • And plenty NLP engines: DialogFlow, Lex, LUIS, Watson, NLP.js
- 33. It all depends • What are your priorities – Open source? – Free? – Programming language of the bot? – Natural languages (e.g. Catalan) – Availability of connectors – On-premises option? – Plain textual bots or fully functional ones? – Low-code/ no-code or tech skills? – Possibility to migrate from one NLP engine to another? – Monitoring dashboard / reports ? – …
- 34. Building bots in Xatkit
- 35. Xatkit’s philosophy: Do not reinvent the wheel Other solutions Plenty of great NLU engines and libraries, let’s not waste our time building a new one but take the best of breed and focus on features to help you create better bots Xatkit is a chatbot orchestration platform
- 36. Another reason to avoid strong coupling to specific NLU/NLP libraries
- 37. Xatkit architecture Chatbot Designer Intent Recognition NLU Providers (platform-specific) Platform DSL / Libraries Intent DSL Xatkit DSL Chatbot User Instant Messaging Platforms Xatkit Runtime Execution DSL uses uses Platform-independent chatbot definition External Services Deployment Configuration Platform Designer
- 38. Xatkit DSL • Bots are created with the Xatkit DSL offering a chatbot- specific syntax for creating: – Intents the bot needs to match – The behavior to execute in response to the matched intents • Our DSL aims to combine the benefits of: – Having a chatbot-specific DSL that facilitates the creation of chatbot applications by providing higher-level chatbot modeling primitives – The full power of a general programming language In this scenario, Internal DSLs are the way to go (Martin Fowler)
- 39. Our DSL is implemented as a Java Fluent API • Create bots using your preferred Java editor – Benefit from all existing Java tooling when developing bots (e.g. debuggers) – Reuse any Java library for complex bot behaviors – Intuitive Fluent Interfaces to help you create advanced conversations – Based on state machine semantics to build any type of bot (there is also a no-code interface, e.g. using excel, for non-tech people)
- 40. I18n: Xatkit helps you build bots ready to be localized
- 41. Easy deployments A Xatkit bot can be packaged as a single jar with no external dependencies. Just execute the jar to get the bot up and running Via a simple configuration file you can connect your bot with external services, e.g. NLP Engines and communication channels
- 42. Xatkit is available on GitHub
- 43. To know more: xatkit.com / info@xatkit.com Xatkit is your best bet to sell more doing less @xatkit – xatkit.com
- 44. Let’s play!!!
Editor's Notes
- #5: We’re still young
- #6: We believe we had good arguments to decide that the world needed another chatbot development platform
- #7: This is really designed to be a hands-on tutorial. We’ll start by installing Xatkit
- #9: AI and Virtual assistants are everywhere
- #10: And the chatbot market is growing non-stop
- #14: Typically, this voice interface ends up being a speech-to-text component so a voicebot is a chatbot plus this speech-to-text and text-to-speech layer on top
- #18: Google on Tuesday unveiled its latest language breakthrough, the development of a conversational language model called LaMDA (Language Model for Dialogue Applications). Google discussed the new model during the keynote address at its I/O conference. Like other recently-developed language models, including BERT and GPT-3, LaMDA is built on Transformer, the neural network architecture that Google Research invented and open-sourced in 2017. However, unlike other language models, Google's LaMDA was trained on dialogue, teaching it how to engage in free-flowing conversations. This training taught LaMDA to deliver responses that not only make sense given the context but are specific.
- #20: By complexity of NL, I mean that beyond intent classification, you’ve plenty of additional issues to deal with in NL: trolls, typos, sentiment analysis
- #21: When you start looking at what you need to build this bot you realize there are many more components involved Event webhooks
- #24: We´ll assume you’re familiar with other complications such as APIs, webhooks,…
- #27: The network uses synonyms and stemming and lemmatization to classify the intent Sometimes we may need to help the Engine by putting close questions in different paths
- #31: Hay mucho “graciosillo” suelto, no queremos que el bot aprenda palaborats
- #33: La diferencia entre un motor NLP y una plataforma es que el motor NLP está especializado en clasificar el texto del usuario, no tanto en lo qué el bot deber hacer con ella.