Text mining lab (summer 2017) - Word Vector Representation
1 like106 views
The document discusses the knowledge discovery process and provides insights into representations of data, including one-hot and distributed representations. It highlights various resources and guides for implementing neural network models and APIs, particularly for word2vec, LSTM, CNN, and RNN. Additional links are provided for further reading and tools related to the subject matter.
![Motel = [0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]
Hotel = [0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0]
●
●
One-hot representation
8](https://image.slidesharecdn.com/textmininglabsummer20171-170725035721/85/Text-mining-lab-summer-2017-Word-Vector-Representation-8-320.jpg)
![hotel = [0.728 0.234 -0.23 0.223]
Distributed representation (low-dimension vector)
9](https://image.slidesharecdn.com/textmininglabsummer20171-170725035721/85/Text-mining-lab-summer-2017-Word-Vector-Representation-9-320.jpg)
![13
P(the|over)
P(fox|over)
P(jumped|over)
P(the|over)
P(lazy|over)
P(dog|over)
P(VOUT
| VIN
)
How to define this prob. distribution?
Determines similarity in [-1,1]
Get a probability in [0,1] out of a similarity in [-1,1]](https://image.slidesharecdn.com/textmininglabsummer20171-170725035721/85/Text-mining-lab-summer-2017-Word-Vector-Representation-13-320.jpg)
Text mining lab (summer 2017) - Word Vector Representation
- 1. Summer 2017 Elvis Saravia PhD, Information Systems and Applications ellfae@gmail.com Github username: omarsar Questions: sli.do (#Z217)
- 2. 2
- 3. ● ● ● ● ● ● ● Knowledge Discovery (KDD) Process 3
- 4. 4
- 5. 5
- 6. ConceptNet 6
- 7. ● ● ● 7
- 8. Motel = [0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0] Hotel = [0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0] ● ● One-hot representation 8
- 9. hotel = [0.728 0.234 -0.23 0.223] Distributed representation (low-dimension vector) 9
- 11. 11 Paper source: https://arxiv.org/pdf/1301.3781.pdf Feedforward Neural Net Language Model (NNLM) variables to optimize denotes window range
- 12. 12
- 13. 13 P(the|over) P(fox|over) P(jumped|over) P(the|over) P(lazy|over) P(dog|over) P(VOUT | VIN ) How to define this prob. distribution? Determines similarity in [-1,1] Get a probability in [0,1] out of a similarity in [-1,1]
- 14. 14
- 16. 16
- 17. ● https://goo.gl/ppHX65 ● ○ Gensim guide for word2vec: https://goo.gl/i2UrdH ● https://goo.gl/7b72S9 ● ● https://goo.gl/uNJDrs ● 17
- 18. 18
- 19. 19
- 20. 20
- 21. 21
- 22. 22
- 23. 23
- 25. a. Build API: (Flask/Django recommended) b. Pretrained models: (Guide: https://goo.gl/5qt2Ki) c. Visualization: d3js / plotly / tensorboard a. LSTM - (Guide: http://colah.github.io/posts/2015-08-Understanding-LSTMs/) b. CNN - (Guide: https://goo.gl/PgLUs7) c. RNN - (Guide: https://goo.gl/5L9kci a. Starting point:https://rare-technologies.com/word2vec-tutorial#app 25