Machine learning for NLP
2 likes3,556 views
The document discusses deep learning and its applications in natural language processing (NLP), outlining its evolution and the significance of neural networks with hidden layers. It covers various NLP tasks, such as speech recognition and text generation, emphasizing the importance of word representations and language models. Additionally, it highlights the advances in automated image captioning and shared representations in machine learning.
![DISTRIBUTED REPRESENTATION
• city = [-0.5, 0.3, …, 0.7]
• town = [-0.52, 0.35, …, 0.8]](https://image.slidesharecdn.com/machinelearningtalk-150518142539-lva1-app6891/85/Machine-learning-for-NLP-30-320.jpg)
Machine learning for NLP
- 1. DEEP LEARNING FOR NLP
- 3. WHAT IS DEEP LEARNING?
- 4. IT’S BASICALLY A NEURAL NETWORK…
- 5. BUT WITH LOTS OF HIDDEN LAYERS
- 7. NO
- 8. IT’S ACTUALLY FROMTHE ‘40S
- 10. AS WELL AS SOME NEW IDEAS e.g. Convolutional Neural Nets
- 11. WHY DO WE NEED NLP
- 13. LOOK AT JAVASCRIPT Developed by some of the most innovative developers
- 14. BULGARIAN ~1300 years > 7 million just today
- 16. LANGUAGE MODELS
- 17. WORD FREQUENCIES • wow, a word cloud
- 18. N-GRAM MODELS
- 21. APPLICATIONS • Spelling correction • Optical Character Recognition • Speech Recognition
- 22. TEXT GENERATORS not very useful, but fun
- 23. HACKERNEWSTITLES • How Facebook is killing Linux on the desktop • Facebook claims it can read your e-mail without a data plan • Only a few countries are teaching children how to drive customers away
- 25. WORD CLASSES • 2014 was a good year. • <YEAR> was a good year.
- 26. NOT GOOD ENOUGH • I like cake • I love pie
- 28. The lecturer criticised the person.
- 29. NEED A REPRESENTATION OF MEANING
- 30. DISTRIBUTED REPRESENTATION • city = [-0.5, 0.3, …, 0.7] • town = [-0.52, 0.35, …, 0.8]
- 33. DEMOTIME
- 37. DEMOTIME AGAIN
- 39. FEEDFORWARD NEURAL NETWORK BASED LANGUAGE MODEL It even sounds fancy
- 40. FEEDFORWARD NEURAL NETWORK BASED LANGUAGE MODEL
- 41. LOCAL CONTEXT ONLY Yesterday - the third day of the month - I went out.
- 43. WHERE DO WE GO NOW?
- 44. PARAGRAPHVECTORS
- 45. WHY DEFINE WORDS AT ALL? Can’t we learn from raw data like the image nets do?
- 48. CAN WE PUT OTHERTHINGS INTHE SAME SPACE?
- 49. APPARENTLY,YES
- 53. DEMO YEAH,AGAIN
- 54. TRIVIA
- 55. “WHAT I LEARNED FROM COMPETING AGAINST A CONVNET ON IMAGENET”
- 59. IT’S A PANDA
- 60. AND NOW?
- 63. REFERENCES • http://colah.github.io/posts/2014-07-Conv-Nets-Modular/ • http://www.fit.vutbr.cz/~imikolov/rnnlm/thesis.pdf • http://cs.stanford.edu/~quocle/paragraph_vector.pdf • http://arxiv.org/pdf/1502.01710v2.pdf