Deep Learning and Modern NLP
1 like373 views
The document outlines key topics in deep learning and natural language processing (NLP), including binary and multi-class text classification, recurrent neural networks (RNNs), and sequence-to-sequence modeling. It discusses various methods such as the perceptron for text classification, the use of LSTM networks, and the significance of word embeddings. Additionally, it explores applications like part-of-speech tagging, named entity recognition, and machine translation.
![Sequence-to-Sequence Inference
In [ ]:](https://image.slidesharecdn.com/deeplearningmodernnlppydatamiami2019-190114142320/85/Deep-Learning-and-Modern-NLP-50-320.jpg)
Deep Learning and Modern NLP
- 1. Deep Learning and Modern Natural Language Processing Zachary S. Brown
- 2. Outline 0. NLP Problem Structure 1. Text Classi cation and the Perceptron 2. Vectorization and Classi cation with RNNs 3. POS Tagging with RNNs 4. Sequence to Sequence Modeling
- 5. Binary Document Classi cation
- 6. Multi-class Document Classi cation
- 7. Multi-class Sequence Classi cation
- 8. Starting Easy: Neural Net with Traditional Vectorization
- 9. Text Classi cation and the Perceptron
- 10. Topics The perceptron and neural network optimization Example
- 11. The Perceptron
- 12. Weights
- 13. Forward Pass
- 14. Loss
- 16. Update Weights
- 18. Vectorization and Classi cation with RNNs
- 19. Topics RNN Encoding for Text Classi cation Recurrent Neural Networks Word Embeddings Example
- 20. Multi-class Document Classi cation
- 21. Text Vectorization with Recurrent Neural Networks
- 22. Recurrent Neural Networks Images graciously sourced from by Christopher OlahUnderstanding LSTM Networks
- 23. Recurrent Neural Networks (unrolled)
- 24. Vanilla Recurrent Neural Networks
- 25. Long Short-term Memory (LSTM) Networks For a deeper dive into the necessity for and implementation of LSTM networks, see by Christopher OlahUnderstanding LSTM Networks
- 26. Encoding a Sequence to a single Vector
- 29. Wait, what about the inputs to the RNN??
- 30. Word Embeddings: Bag-of-Words vs. Dense Representations
- 31. LSTM Text Encoding and Classi cation Example
- 32. LSTMs for Sequence to Sequence Modeling
- 33. Fixed Length Sequence to Sequence
- 34. Use Cases
- 36. Use Cases POS Tagging Named Entity Recognition
- 37. Use Cases POS Tagging Named Entity Recognition Extractive Summarization
- 38. Use Cases POS Tagging Named Entity Recognition Extractive Summarization
- 39. Before: Encoding a Sequence to a single Vector
- 40. Now: Encoding a Sequence to a Sequence of Vectors
- 42. Sequence to Sequence Modeling
- 43. Topics Encoder-Decoder Architecture Neural Machine Translation
- 44. Variable Length Sequence to Sequence
- 49. Sequence-to-Sequence Training: Teacher Forcing