NLP Structured Data Investigation on Non-Text
3 likes530 views
Casey Stella presents on using natural language processing (NLP) techniques like word2vec to analyze structured medical data. Clinical encounters can be viewed as "sentences" with vital signs, labs, procedures, diagnoses and drugs as "words". Word2vec is applied to a dataset of 197,340 clinical records to learn relationships between these clinical concepts without domain expertise. The results are explored using Spark and IPython Notebook on Hadoop.
![Word2Vec
Word2Vec is a vectorization model created by Google [1] that attempts to learn
relationships between words automatically given a large corpus of sentences.
• Gives us a way to find similar words by finding near neighbors in the vector space
with cosine similarity.
1
http://radimrehurek.com/2014/12/making-sense-of-word2vec/
Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016](https://image.slidesharecdn.com/m12thur1210caseystella-160906145311/85/NLP-Structured-Data-Investigation-on-Non-Text-6-320.jpg)
![Word2Vec
Word2Vec is a vectorization model created by Google [1] that attempts to learn
relationships between words automatically given a large corpus of sentences.
• Gives us a way to find similar words by finding near neighbors in the vector space
with cosine similarity.
• Uses a neural network to learn vector representations.
1
http://radimrehurek.com/2014/12/making-sense-of-word2vec/
Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016](https://image.slidesharecdn.com/m12thur1210caseystella-160906145311/85/NLP-Structured-Data-Investigation-on-Non-Text-7-320.jpg)
![Word2Vec
Word2Vec is a vectorization model created by Google [1] that attempts to learn
relationships between words automatically given a large corpus of sentences.
• Gives us a way to find similar words by finding near neighbors in the vector space
with cosine similarity.
• Uses a neural network to learn vector representations.
• Work by Pennington, Socher, and Manning [2] shows that the word2vec model is
equivalent to a word co-occurance matrix weighting based on window distance and
lowering the dimension by matrix factorization.
1
http://radimrehurek.com/2014/12/making-sense-of-word2vec/
Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016](https://image.slidesharecdn.com/m12thur1210caseystella-160906145311/85/NLP-Structured-Data-Investigation-on-Non-Text-8-320.jpg)
![Word2Vec
Word2Vec is a vectorization model created by Google [1] that attempts to learn
relationships between words automatically given a large corpus of sentences.
• Gives us a way to find similar words by finding near neighbors in the vector space
with cosine similarity.
• Uses a neural network to learn vector representations.
• Work by Pennington, Socher, and Manning [2] shows that the word2vec model is
equivalent to a word co-occurance matrix weighting based on window distance and
lowering the dimension by matrix factorization.
Takeaway: The technique boils down, intuitively, to a riff on word co-occurence. See
here1 for more.
1
http://radimrehurek.com/2014/12/making-sense-of-word2vec/
Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016](https://image.slidesharecdn.com/m12thur1210caseystella-160906145311/85/NLP-Structured-Data-Investigation-on-Non-Text-9-320.jpg)
![Bibliography
[1] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of
word representations in vector space. CoRR, abs/1301.3781, 2013.
[2] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global
vectors for word representation. In Proceedings of the 2014 Conference on Empirical
Methods in Natural Language Processing (EMNLP), pages 1532–1543. Association
for Computational Linguistics, 2014.
Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016](https://image.slidesharecdn.com/m12thur1210caseystella-160906145311/85/NLP-Structured-Data-Investigation-on-Non-Text-15-320.jpg)
NLP Structured Data Investigation on Non-Text
- 1. NLP Structured Data Investigation on Non-Text Casey Stella @casey_stella 2016 Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 2. Table of Contents Preliminaries Borrowing from NLP Demo Questions Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 3. Introduction Hi, I’m Casey Stella! Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 4. Domain Challenges in Data Science A data scientist has to merge analytical skills with domain expertise. • Often we’re thrown into places where we have insufficient domain experience. • Gaining this expertise can be challenging and time-consuming. • Unsupervised machine learning techniques can be very useful to understand complex data relationships. Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 5. Domain Challenges in Data Science A data scientist has to merge analytical skills with domain expertise. • Often we’re thrown into places where we have insufficient domain experience. • Gaining this expertise can be challenging and time-consuming. • Unsupervised machine learning techniques can be very useful to understand complex data relationships. We’ll use an unsupervised structure learning algorithm borrowed from NLP to look at medical data. Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 6. Word2Vec Word2Vec is a vectorization model created by Google [1] that attempts to learn relationships between words automatically given a large corpus of sentences. • Gives us a way to find similar words by finding near neighbors in the vector space with cosine similarity. 1 http://radimrehurek.com/2014/12/making-sense-of-word2vec/ Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 7. Word2Vec Word2Vec is a vectorization model created by Google [1] that attempts to learn relationships between words automatically given a large corpus of sentences. • Gives us a way to find similar words by finding near neighbors in the vector space with cosine similarity. • Uses a neural network to learn vector representations. 1 http://radimrehurek.com/2014/12/making-sense-of-word2vec/ Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 8. Word2Vec Word2Vec is a vectorization model created by Google [1] that attempts to learn relationships between words automatically given a large corpus of sentences. • Gives us a way to find similar words by finding near neighbors in the vector space with cosine similarity. • Uses a neural network to learn vector representations. • Work by Pennington, Socher, and Manning [2] shows that the word2vec model is equivalent to a word co-occurance matrix weighting based on window distance and lowering the dimension by matrix factorization. 1 http://radimrehurek.com/2014/12/making-sense-of-word2vec/ Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 9. Word2Vec Word2Vec is a vectorization model created by Google [1] that attempts to learn relationships between words automatically given a large corpus of sentences. • Gives us a way to find similar words by finding near neighbors in the vector space with cosine similarity. • Uses a neural network to learn vector representations. • Work by Pennington, Socher, and Manning [2] shows that the word2vec model is equivalent to a word co-occurance matrix weighting based on window distance and lowering the dimension by matrix factorization. Takeaway: The technique boils down, intuitively, to a riff on word co-occurence. See here1 for more. 1 http://radimrehurek.com/2014/12/making-sense-of-word2vec/ Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 10. Clinical Data as Sentences Clinical encounters form a sort of sentence over time. For a given encounter: • Vitals are measured (e.g. height, weight, BMI). • Labs are performed and results are recorded (e.g. blood tests). • Procedures are performed. • Diagnoses are made (e.g. Diabetes). • Drugs are prescribed. Each of these can be considered clinical “words” and the encounter forms a clinical “sentence”. Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 11. Clinical Data as Sentences Clinical encounters form a sort of sentence over time. For a given encounter: • Vitals are measured (e.g. height, weight, BMI). • Labs are performed and results are recorded (e.g. blood tests). • Procedures are performed. • Diagnoses are made (e.g. Diabetes). • Drugs are prescribed. Each of these can be considered clinical “words” and the encounter forms a clinical “sentence”. Idea: We can use word2vec to investigate connections between these clinical concepts. Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 12. Demo As part of a Kaggle competition2, Practice Fusion, a digital electronic medical records provider released depersonalized clinical records of 10,000 patients. I ingested and preprocessed these records into 197,340 clinical “sentences” using Pig and Hive. 2 https://www.kaggle.com/c/pf2012-diabetes Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 13. Demo As part of a Kaggle competition2, Practice Fusion, a digital electronic medical records provider released depersonalized clinical records of 10,000 patients. I ingested and preprocessed these records into 197,340 clinical “sentences” using Pig and Hive. MLLib from Spark now contains an implementation of word2vec, so let’s use pyspark and IPython Notebook to explore this dataset on Hadoop. 2 https://www.kaggle.com/c/pf2012-diabetes Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 14. Questions Thanks for your attention! Questions? • Code & scripts for this talk available on my github presentation page.3 • Find me at http://caseystella.com • Twitter handle: @casey_stella • Email address: cstella@hortonworks.com 3 http://github.com/cestella/presentations/ Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016
- 15. Bibliography [1] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781, 2013. [2] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Association for Computational Linguistics, 2014. Casey Stella@casey_stella (Hortonworks) NLP Structured Data Investigation on Non-Text 2016