Convolutional Neural Networks for Sentence Classification
Download as PPTX, PDF1 like235 views
This document discusses convolutional neural networks for sentence classification. It begins with introducing automated text classification and definitions of key concepts like deep learning, language models, neural language models, and convolutional neural networks. It then discusses various datasets used for evaluating models. Several CNN models are described and their results on different datasets are presented, outperforming other methods. The document concludes by showing the most similar words learned by static and non-static CNN channels.
![Model
𝑥1:𝑛 = 𝑥1 ⊕ 𝑥2 ⊕ … 𝑥 𝑛
𝑐𝑖 = 𝑓(𝑤. 𝑥𝑖:𝑖+ℎ−1 + 𝑏)
𝐶 = [𝑐1, 𝑐2, … , 𝑐 𝑛−ℎ+1]
𝐎𝐢𝐣 = 𝐟 𝐧𝐞𝐭 𝐢𝐣 𝐧𝐞𝐭 𝐢𝐣 =
𝐢=𝟏
𝐥
𝐰𝐢𝐣
𝟏
. 𝐱 𝐢 + 𝐛
11](https://image.slidesharecdn.com/nlp-finalproject-170420192433/85/Convolutional-Neural-Networks-for-Sentence-Classification-12-320.jpg)
Convolutional Neural Networks for Sentence Classification
- 1. Convolutional Neural Networks for Sentence Classification Students: Minoo Moradi Mahsa Deylam nejad Supervisor: Dr.Nasiri Spring 2017 In the name of GOD
- 2. 01
- 3. Automated text classification is the task of automatically assigning input text to a set of classes. 02
- 4. 01 03 Options Options 02Options Neural Language Model Deep Learning Language Model 03
- 5. 04 06 Options Options 05Options DropOut Curse of dimensionality Convolutional Neural Network 04
- 6. Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. In fact it refers to artificial neural networks that are composed of many layers. 6 ADefinition Deep Learning 05
- 7. A language model is a function, or an algorithm for learning such a function, that captures the salient statistical characteristics of the distribution of sequences of words in a natural language, typically allowing one to make probabilistic predictions of the next word given preceding ones. 7 ADefinition Language Model 06
- 8. A neural language model is a language model based on Neural Networks , exploiting their ability to learn distributed representations to reduce the impact of the curse of dimensionality. 8 ADefinition Neural Language Model 07
- 9. In the context of learning algorithms, the curse of dimensionality refers to the need for huge numbers of training examples so that When the number of input variables increases, the number of required examples can grow exponentially. 9 ADefinition Curse of Dimensionality 08
- 10. Convolutional Neural Networks are very similar to ordinary Neural Networks . they are made up of neurons that have learnable weights and biases. Each neuron receives some inputs, performs a dot product and optionally follows it with a non-linearity. 10 ADefinition Convolutional Neural Network 09
- 11. DropOut is a regularization method that sets to zero the activations of hidden units for each training case at training time. 11 ADefinition DropOut 10
- 12. Model 𝑥1:𝑛 = 𝑥1 ⊕ 𝑥2 ⊕ … 𝑥 𝑛 𝑐𝑖 = 𝑓(𝑤. 𝑥𝑖:𝑖+ℎ−1 + 𝑏) 𝐶 = [𝑐1, 𝑐2, … , 𝑐 𝑛−ℎ+1] 𝐎𝐢𝐣 = 𝐟 𝐧𝐞𝐭 𝐢𝐣 𝐧𝐞𝐭 𝐢𝐣 = 𝐢=𝟏 𝐥 𝐰𝐢𝐣 𝟏 . 𝐱 𝐢 + 𝐛 11
- 13. 𝑦 = 𝑤. 𝑧 + 𝑏 𝑦 = 𝑤. 𝑧 ∘ 𝑟 + 𝑏 12 Regularization
- 14. 01 Option 02 Option 04 Option 03 Option SST-1 Text Text 0605 07 Option Option Option MR SST-2 TREC Subj MPQA CR DataSets
- 15. Data C l N |𝑽| |𝑽 𝒑𝒓𝒆| Test MR 2 20 10662 18765 16448 cv SST-1 5 18 11855 17836 16262 2210 SST-2 2 19 9613 16185 14838 1821 Subj 2 23 10000 21323 17913 cv TREC 6 10 5952 9592 9125 500 CR 2 19 3775 5340 5046 cv MPQA 2 3 10606 6246 6083 cv 14
- 18. Most similar words Static channel Non statistic channel bad Good Terrible Horrible Lousy Terrible Horrible Lousy stupid good Great Bad Terrific Decent Nice Decent Solid Terrific N’t os ca never ireland wo not never nothing neither ! 2,500 entire jez changer 2,500 lush beautiful terrific , decasia abysmally demise valiant but dragon a and 17
- 19. Conclusion 18
- 20. Thank you