Introduction to Natural Language Processing
Download as PPTX, PDF2 likes556 views
The document introduces natural language processing (NLP), explaining its importance in enabling applications to understand human language. It discusses two primary approaches: deep learning, which teaches computers language, and algorithmic NLP, which uses heuristics to yield approximate results. Various NLP techniques, such as tokenization and sentiment analysis, are outlined, along with their statistical limitations and language dependence.
Introduction to Natural Language Processing
- 1. an introduction to Natural Language Processing (NLP)
- 2. Who am I? • David Rostcheck • I’m a consulting data scientist • Follow my articles on LinkedIn
- 3. Who am I? • Jennifer Barry • I do research and project management
- 4. what is Natural Language Processing and why do we need it?
- 6. so our applications are much more useful when they understand language
- 7. and can help us.
- 8. but the problem is… computers don’t really understand language
- 9. What can we do?
- 10. solution 1: Deep Learning idea: teach computers language (like we teach it to people) (these are model-free methods)
- 11. solution 2: Natural Language Processing (NLP) via algorithms idea: fake it! find algorithms that give approximately correct results most of the time (these are heuristics)
- 12. Today, we learn about faking it ;)
- 14. They are language-dependent (you need a new algorithm for each language) algorithm-one algoritmo-dos Algorithmus-drei
- 15. They are ---------- fast!
- 16. easy to implement … at first, anyway...
- 17. They are computationally efficient, so they scale very well! | | | | | | | | | | | | | | |
- 18. they produce results that are mostly correct most of the time
- 19. buuuuut....
- 20. NLP relies on statistics so it is only statistically true
- 21. Without real understanding, accuracy is limited and cannot easily be improved NLP: looks fine to me!
- 22. NLP techniques
- 23. stemming learn-ed learn-er learn learn-ing removes suffixes – discards nuance, emphasizes root concept
- 24. tokenization Turns text into a stream of sentences or words or syllables
- 25. This is simple in some languages …but not so simple in others
- 26. bag of words transforms a document into a collection of unordered words
- 27. stripping removes low-information signals (common words, punctuation)
- 28. n-gram: a sequence of n tokens taken together
- 29. tf/idf (“term frequency over inverse document frequency”) Finds unusual words that appear often in this document (used in search engines)
- 30. examples
- 32. Flesch-Kincaid Grade Level formula:
- 33. Topic extraction
- 34. Personality Analysis (using IBM Watson’s Personality Insights service)
- 35. social network analysis from text
- 36. Sentiment analysis (using Turi GraphLab Create)
- 37. Questions? Contact: drostcheck@leopardllc.com, twitter: @davidrostcheck Articles: http://linkedin.com/in/davidrostcheck