God Mode for designing scenario-driven skills for DeepPavlov Dream
0 likes57 views
The document summarizes the capabilities and development of DeepPavlov, an open-source conversational AI platform. It discusses DeepPavlov's God Mode for building open-domain bots using a discourse-driven integrated dialog development environment. DeepPavlov utilizes speech functions and dialog acts to enable more predictable dialog design and improves over regular expressions and intents. The platform provides tools like a VSCode extension to help with rapid dialog design and skills development.
![DeepPavlov.ai
Scenario-Driven approach enables sequential experience and is strategic, yet it takes a long time and still misses
on what user can say.
User
•Let’s chat
Bot
•(greeting)
(question)
User
•[utterance]
Bot
•[utterance]
User
•[utterance]
Bot
•[utterance]
User
•[utterance]
Bot
•[utterance]
Small Talk Movies Skill
Pros: Scenario-Driven skills enabled us to give users several conversation turns with
the same context, and limited user understanding within human-curated scenarios
Cons: given that we can’t predict what user might say, lots of iterations needed.](https://image.slidesharecdn.com/s41406danielkornevdd-idde-godmodefordesigningscenario-drivenskillsfordeeppavlovdream-final-220330131549/85/God-Mode-for-designing-scenario-driven-skills-for-DeepPavlov-Dream-7-320.jpg)
God Mode for designing scenario-driven skills for DeepPavlov Dream
- 1. GOD MODE FOR BUILDING OPEN- DOMAIN BOTS WITH DISCOURSE- DRIVEN INTEGRATED DIALOG DEVELOPMENT ENVIRONMENT Daniel Kornev CPO @ DeepPavlov
- 2. Grand Challenge: create a socialbot that can engage in a fun, high quality conversation on popular societal topics for 20 minutes and achieve an average rating of at least 4.0/5.0. Alexa Prize 3 vs 4 Winners: 1. $500K, 3.8 vs 3.28, 7 min 32 sec vs 14 min 14 sec 2. $100K 3.17 vs 3.25 Alexa Prize 4 Winner: Alquist (2nd in ‘17, 3rd in `18,`20) – Czech Technical University 2
- 3. DeepPavlov.ai “The aim is to identify a way of looking at and manipulating commonsense knowledge that is consistent with and can support what we consider to be the two most fundamental aspects of intelligent cognitive behaviour: the ability to learn from experience and the ability to reason from what has been learned. We are therefore seeking a semantics of knowledge that can computationally support the basic phenomena of intelligent behaviour.” Leslie Valiant
- 4. 1000+ Commits 800+ PRs ~30 skills 10+ annotators 1 PhD! DeepPavlov.ai
- 6. DeepPavlov.ai pip install Python-Native Lots of extensions Battle-tested Entity Detection Extensions Intent Classifier Dialog Acts & Speech Functions Dialog Analysis Dashboard
- 7. DeepPavlov.ai Scenario-Driven approach enables sequential experience and is strategic, yet it takes a long time and still misses on what user can say. User •Let’s chat Bot •(greeting) (question) User •[utterance] Bot •[utterance] User •[utterance] Bot •[utterance] User •[utterance] Bot •[utterance] Small Talk Movies Skill Pros: Scenario-Driven skills enabled us to give users several conversation turns with the same context, and limited user understanding within human-curated scenarios Cons: given that we can’t predict what user might say, lots of iterations needed.
- 8. DeepPavlov.ai Problem Designing Open-Domain Chatbots is a complex process. User can say anything Interpretation How to recognize pattern? Response How to react? Bot: Hi, how are you doing? User: Hi! Regular Expressions: Require detailed scenarios and long iterative development process Intents: Work only within the given domain (task-oriented chatbots) or at high-level (greeting, etc.) Dialog Acts: Work in limited cases (yes/no, question/statement) User: Hi! User: What’s up? User: I’m sad. User: Had an awful convo. User: Switch off lights
- 9. DeepPavlov.ai There are at least 2 ways* look at a Conversation: * S. Eggins & D. Slade, Analysing Casual Conversation.London: Cassell, 1997 Pragmatic Conversation • Motivated by clear pragmatic purpose. Aka task- oriented. Usually very short. Formal. Casual Conversation • NOT motivated by clear pragmatic purpose. Can and often are lengthy. Informal, can have humor. Aka chit-chat.
- 10. DeepPavlov.ai There are several ways* to analyze a Casual Conversation: * S. Eggins & D. Slade, Analysing Casual Conversation. London: Cassell, 1997
- 11. There are at least 4 different approaches to classify utterances & sentences: Speech Acts* • Work at utterance level. Hearer interprets speaker’s intentions and tries to interpret desired actions from hearer. Dialog Acts • Work at sentence level. Ascribe each sentence’s dialog function to the entire utterance. Speech Functions • Work at sentence level. Similar to Speech Acts but they produce utterance’s through its role in Discourse. Utterance Acts** • Work at utterance level but include body movements. **Not applicable for us as we can’t see the person *Original authors were not concerned with Discourse
- 12. DeepPavlov.ai There are many taxonomies for each of the classification types: Speech Acts • Austin (1962) • Searle (1975) • Bach and Harnish (1979) Dialog Acts • Jurafsky (1997) – SWBD-DAMSL • MIDAS (2019) Speech Functions • Halliday (1984, 1994) • Eggins and Slade (1997) Speech Acts & Dialog Acts: Speech Acts == Dialog Acts (Jurafsky, 2019) Speech Acts != Dialog Acts (Webb, 2010) vs. Speech Acts & Speech Functions: Speech Acts = interpretation (logico-phil.) Speech Functions = production of utterances (discourse)
- 13. MIDAS has two groups of Dialog Acts: Semantic and Functional. Gunrock’s Report for Alexa Prize 2 (2018)
- 14. MIDAS has two groups of Dialog Acts: Semantic and Functional. Gunrock’s Report for Alexa Prize 2 (2018)
- 15. DeepPavlov.ai Eggins and Martin (1997) Discourse structure patterns operate across turns: thus overtly interactional & sequential Discourse Management is a basis for acting across turns, thus becoming strategic
- 16. Eggins and Slade (1997) Speech Functions control Discourse: Give information Demand information Speech Acts Discourse Moves Speech Function Example: open:initiate:give_opinion
- 17. Eggins and Slade (1997) Speech Functions have hierarchy based on the role in Discourse: move open attend command Initiate Give Fact opinion Demand Open Fact Opinion Closed Fact Opinion sustain Continue Monitor Prolong Elaborate Extend Enhance Append Elaborate Extend Enhance React Respond Support Develop Elaborate Extend Enhance Engage Register Reply Accept Comply Agree Answer Acknowledge Affirm Confront Disengage Reply Decline Non-comply Disagree Withold Disawow Contradict Rejoinder Support Track Check Confirm Clarify Probe Response Resolve Repair Acquiesce Confront Challenge Detach Rebound Counter Response Unresolve Refute Re-challenge
- 18. Eggins and Slade (1997) Removing SFs we don’t have to classify from user’s utterances in Alexa Prize move open attend Initiate Give Fact opinion Demand Open Fact Opinion Closed Fact Opinion sustain Continue Monitor Prolong Elaborate Extend Enhance Append Elaborate Extend Enhance React Respond Support Develop Elaborate Extend Enhance Engage Register Reply Accept Comply Agree Answer Acknowledge Affirm Confront Disengage Reply Decline Non-comply Disagree Withold Disawow Contradict Rejoinder Support Track Check Confirm Clarify Probe Response Resolve Repair Acquiesce Confront Challenge Detach Rebound Counter Response Unresolve Refute Re-challenge
- 19. DeepPavlov (2021) (removed SFs for commands, discussing physical goods, non-verbal, multiple people) move open Attend Command Initiate Give Fact opinion Demand Open Fact Opinion Closed Fact Opinion sustain Continue Monitor Prolong Elaborate Extend Enhance React Respond Support Develop Elaborate Extend Enhance Engage Register Reply Agree Acknowledge Affirm Confront Reply Disagree Disawow Contradict Rejoinder Support Track Check Confirm Clarify Probe Response Resolve Confront Challenge Detach Rebound Counter Response Unresolve Refute Re-challenge
- 20. DeepPavlov.ai We used our Speech Functions Classifier as well as Dialog Acts Classifier (MIDAS) made by Gunrock (Alexa Prize 3 team) to measure the effect
- 21. Original Book Skill was designed during Alexa Prize 3 and rewritten in DFF by one of our interns. ? ?
- 22. DeepPavlov.ai Step 1 The question: could use of Speech Functions and/or MIDAS make dialog more predictable for dialog designer? Let people talk to the bot with only this skill enabled Step 2 Classify utterances w/ Speech Functions & MIDAS Step 3 Run Speech Functions & MIDAS predictors Step 4 Compute # of times predictors predicted user utterance classes correctly & not Step 5 Compare results with vanilla data
- 23. DeepPavlov.ai Primitive but working approach: ignorance, yes/no, custom conditions.
- 24. DeepPavlov.ai Comparing original approach to MIDAS:
- 25. DeepPavlov.ai Manual Labeling vs. Classifier:
- 27. DeepPavlov.ai Comparing original approach to SFs:
- 28. DeepPavlov.ai Manual Labeling vs. Classifier:
- 29. Speech Functions Predictor shows the potential of the Speech Functions-based scenario-driven skill
- 30. DeepPavlov.ai For Dialogue Designers for Doom Players God Mode for
- 31. DeepPavlov.ai Extension for VS Code Open-Source Dev Tool Draw.io-based Draw.io <> Python DSL of DFF With Discourse Recommendations… Aids in Rapid Dialog Design Statistics… Plugins!
- 33. DeepPavlov.ai +
- 34. DeepPavlov.ai Custom spatial UI, re-built suggestions, auto response generation and… a new name!
- 54. DeepPavlov.ai Multiskill orchestration Conversa- tionalskills NLP frameworks ML platforms Proprietary Open Source ▪ Multiskill Orchestration • DeepPavlov Agent is an engine for conversational skill deployment and orchestration ▪ Conversational Skills • DeepPavlov Dream - collection of pre-build conversational skills and a default AI Assistant distribution package • DeepPavlov DFF - framework for building scenario-driven chatbots & Dream skills ▪ NLP Frameworks • DeepPavlov Library - pretrained models and both Python & declarative approaches for building NLP pipelines ▪ ML Platforms • TensorFlow and PyTorch as backends
- 55. DeepPavlov.ai DeepPavlov DF Designer DeepPavlov DF SDK DeepPavlov DF Engine DeepPavlov Annotators Library, Agent, Annotators, DF Engine, DF SDK, DF Designer, Dream Builder, Dream + Dashboards (DF + Dream) DeepPavlov Dream Builder DeepPavlov DF Dashboard DeepPavlov Dream Dashboard DeepPavlov DFF
- 56. DeepPavlov DF Designer DeepPavlov DFF DeepPavlov DF Engine DeepPavlov Annotators DeepPavlov Dream Builder Dev Tools Chatbot & AI Assistant Platforms Engines Models DeepPavlov DF Dashboard DeepPavlov Dream Dashboard DeepPavlov DF SDK
- 57. SPECIAL THANKS! Denis Kuznetsov DFF Team Lead Dmitry Evseev Wiki for DFF SDK Lida Ostyakova Speech Functions Bálint Magyar VS Code Extension Dmitry Babadeev VS Code Extension Ksenia Petyxova Skill Writer Nika Smilga Skill Writer Oleg Serikov VS Code Extension Maria Molchanova Product Manager Daniel Kornev Deputy Head of Lab Mikhail Burtsev Head of Lab Yuri Kuratov Senior Researcher Maxim Talimanchuk Dev Ops Engineer Fedor Ignatov Senior SW Engineer Dilyara Baymurzina Dream Team Lead
- 58. bit.ly/df-designer or DF Designer in VS Code Marketplace DF Designer: bit.ly/dff-sdk DF SDK: bit.ly/df-designer-qa or use QR code: Talk to us about DF Designer: Come join us! medium.com/deeppavlov Read us: forum.deeppavlov.ai Talk to us: @DeepPavlovDreamDiscussions TG: @DeepPavlov Twitter/TG: