NLP 101 + Chatbots
- 1. NLP 101 + ChatBots
- 2. HELLO! Chris Shei Tech Evangelist @ Jet.com
- 3. What will you learn today? ● NLP 101 (What it is / Difficulties) ● Turing Test ● Language Modeling / Feature Learning Techniques (word2vec) ● Dialog Flow Demo (Intents, Entities, etc.)
- 5. What is Natural Language Processing? Subfield of AI + Linguistics focused on the interaction between computers and human natural languages
- 6. Source: Bill MacCartney 2014
- 7. Why is NLP difficult? 2
- 8. Ambiguity in Language Miscommunication happens even when it’s just two humans communicating… Humans can learn and clarify
- 9. Examples of ambiguity in just the English Language
- 10. Ambiguity in Speech Recognition “Recognize Speech” vs “Wreck a Nice Beach” We ‘mishear’ lyrics all the time…
- 11. Ambiguity in Pronunciation (Heteronyms) Heteronyms - Words spelled identically, but have different meanings when pronounced differently. Lead me to the lead (metal). My next project is to learn how to project my voice.
- 12. Syntactic Ambiguity (ambiguous sentence structure) “I saw a man on the hill with a telescope” Source: Deniz Yuret
- 13. Lexical Ambiguity Polysemy - more than one meaning “Picture (verb) a high mountain” “That’s a nice picture (noun)” Homonomy - same pronunciation + spelling, different in meaning Lie - making an untrue statement Lie - to lay down
- 14. Lexical Ambiguity Will, will Will will Will Will's will? Will(1) Will (3) Future tense Bequeath (to)Will (2) Will (2) Document Someone asked Will 1 directly if Will 2 plans to bequeath his own will, the document, to Will 3.
- 15. So why is NLP cool?
- 16. “There’s no way you can have an AI system that’s humanlike that doesn’t have language at the heart of it - Josh Tenenbaum
- 17. Turing Test 3
- 18. Test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. Source: Wikipedia
- 20. “Are there imaginable digital computers which would do well in the imitation game? - Alan Turing
- 21. The Original Imitation Game
- 22. The Modern Turing Test Judge to determine if they’re speaking to a human or computer Source: Wikipedia
- 23. ‘Passing’ the Turing Test Established by tricking the judges: - Building a ‘forgiving’ persona (young or ESL) - Evading questions - Deflecting questions back to the user
- 24. Eugene Goostman ‘passes’ the Turing Test Scott: Which is bigger, a shoebox or Mount Everest? Eugene: I can’t make a choice right now. I should think it out later. And I forgot to ask you where you are from… Scott: How many legs does a camel have? Eugene: Something between 2 and 4. Maybe, three? :-))) By the way, I still don’t know your specialty – or, possibly, I’ve missed it? Source: Scott Aaronson
- 25. Imitation Game Criticisms What is 12345*54321? (Intelligent or unintelligent behavior?) Naivete + training of interrogators (What if judges think a human is a machine?) Impracticality + Irrelevance (better and more valuable tests of intelligence)
- 27. Early NLP Systems: Rule-based - Hard If-Then rules (e.g. Decision Trees) Later NLP Systems: Statistical models - Soft, probabilistic decisions Benefits of later NLP systems: - Automatic focus on the most ‘common’ cases (instead of deciding where to focus) - Robust to unfamiliar input (e.g. misspelled or accidentally omitted words) - Accuracy can be improved by providing more input data (easier than increasing the complexity of a system)
- 28. How do we encode words?
- 29. One-hot encoding | Disadvantages? Source: Marco Bonzanini
- 30. Word Embeddings By encoding them as a vector, we can apply mathematical rules to them
- 31. Let’s do some word math: King - Man + Woman = ? Queen
- 33. Word Vectors Source: Jayesh Bapu Ahire
- 34. Source: Jayesh Bapu Ahire
- 35. So how do we get these vectors?
- 36. Word Embedding (e.g. word2vec) Given an ‘input’ word, tell us the probability of these other words showing up nearby Source: Jayesh Bapu Ahire
- 37. Word Embedding (e.g. word2vec) Source: Chris McCormick
- 38. “ You shall know a word by the company it keeps - J.R. Firth Distributional Hypothesis
- 39. Standard NLP Workflow Resources: Ultimate guide to understand & implement NLP A practitioners guide to NLP How to solve 90% of NLP Problems Source: Dipanjan Sarkar
- 40. DialogFlow 5
- 41. DialogFlow Build engaging voice + text based conversational interfaces supported by a powerful NLU engine
- 42. What are we going to build today Ask for the address Order a T-Shirt - Color - Size Asks if you want a receipt - Asks for an email if yes - Gives a confirmation # if no
- 43. Intent - a specific action the user is trying to achieve
- 44. What are we going to build today Ask for the address Order a T-Shirt - Color - Size Asks if you want a receipt - Asks for an email if yes - Gives a confirmation # if no
- 45. Intents - Intent Name - Training Phrases - Response
- 46. Default Intents | Fallback Intent + Welcome Intent
- 47. What are we going to build today Ask for the address Order a T-Shirt - Size - Color Asks if you want a receipt - Asks for an email if yes - Gives a confirmation # if no
- 48. What are we going to build today Ask for the address Order a T-Shirt - Size (Required) - Color (Required) Asks if you want a receipt - Asks for an email if yes - Gives a confirmation # if no
- 49. Actions + Parameters Extracts relevant information from user utterances (e.g. dates, times, names, etc)
- 50. Entities Entities are Dialogflow's mechanism for identifying and extracting useful data from natural language inputs.
- 51. What are we going to build today Ask for the address Order a T-Shirt - Size (Required) - Color (Required) Asks if you want a receipt - Asks for an email if yes - Gives a confirmation # if no
- 52. What are we going to build today Ask for the address Order a T-Shirt - Size (Required) - Color (Required) Asks if you want a receipt - Asks for an email if yes - Gives a confirmation # if no
- 53. Follow Up Intents + Context
- 55. The docs are fantastic! https://dialogflow.com/docs
- 56. THANKS! Any questions? You can find me at chris.shei@jet.com @ckshei www.bespokelife.co
Editor's Notes
- #7: Source: https://nlp.stanford.edu/~wcmac/papers/20140716-UNLU.pdf
- #11: Read More: https://www.slideshare.net/zareen/challenges-in-nlp
- #13: Source: http://www.denizyuret.com/2010/12/research-focus.html
- #17: Josh Tenenbaum Professor of cognitive science and computation at MIT
- #18: https://www.csee.umbc.edu/courses/471/papers/turing.pdf https://www.youtube.com/watch?v=3wLqsRLvV-c http://www.psych.utoronto.ca/users/reingold/courses/ai/turing.html https://en.wikipedia.org/wiki/Turing_test
- #19: https://en.wikipedia.org/wiki/Turing_test
- #20: Question is too vague… ‘thinking’ is difficult to define https://en.wikipedia.org/wiki/Turing_test#Imitation_game_vs._standard_Turing_test
- #21: Exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human
- #23: https://en.wikipedia.org/wiki/Turing_test
- #25: Source: https://www.scottaaronson.com/blog/?p=1858
- #26: https://en.wikipedia.org/wiki/Turing_test
- #28: Source: https://en.wikipedia.org/wiki/Natural_language_processing
- #30: Source: https://www.pycon.it/media/conference/slides/word-embeddings-for-natural-language-processing-in-python.pdf
- #33: https://blogs.mathworks.com/loren/2017/09/21/math-with-words-word-embeddings-with-matlab-and-text-analytics-toolbox/
- #34: Source: https://medium.com/@jayeshbahire/introduction
- #35: Source: https://medium.com/@jayeshbahire/introduction
- #40: https://www.analyticsvidhya.com/blog/2017/01/ultimate-guide-to-understand-implement-natural-language-processing-codes-in-python/ https://towardsdatascience.com/a-practitioners-guide-to-natural-language-processing-part-i-processing-understanding-text-9f4abfd13e72 https://blog.insightdatascience.com/how-to-solve-90-of-nlp-problems-a-step-by-step-guide-fda605278e4e