Introduction to spaCy
0 likes1,683 views
This document introduces spaCy, an open-source library for advanced natural language processing. It describes spaCy as the fastest NLP library in the world, with a simple API and ability to integrate with deep learning frameworks. It provides examples of how to quickly install spaCy and use it to perform part-of-speech tagging, dependency parsing, and named entity recognition on sample text. It also introduces textacy, a library built on spaCy that enables higher-level NLP tasks and text analysis.
![うちの研究室的に便利そうな機能
textacy.extract.pos_regex_matches
from textacy.extract import pos_regex_matches
from textacy.constants import POS_REGEX_PATTERNS
list(pos_regex_matches(nlp('Can you process this text?'),
POS_REGEX_PATTERNS['en']['NP']))
[this text]
16 / 17](https://image.slidesharecdn.com/newpresentation2017-10-181842561-171019010753/85/Introduction-to-spaCy-16-320.jpg)
Introduction to spaCy
- 3. What's spaCy? "Industrial-Strength" NLP Library Fastest in the world written in Cython Get things done easy to install simple API Deep learning interoperates seamlessly with TensorFlow, Keras, Scikit-Learn, Gensim 3 / 17
- 4. Fastest in the world Syntactic parsing (Choi et al., IJCNLP, 2015) https://spacy.io/docs/api/ 4 / 17
- 5. Fastest in the world Detailed speed comparison https://spacy.io/docs/api/ 5 / 17
- 6. Get things done Installation: $ pip install spacy $ python -m spacy download en Load model and process text: import spacy nlp = spacy.load('en') doc = nlp('Can you process this text?') 6 / 17
- 7. Get things done POS tagging: for token in doc: print(token, token.pos_) Can VERB you PRON process VERB this DET text NOUN ? PUNCT 7 / 17
- 8. Get things done Dependency parsing: for token in doc: print('{} -({})-> {}'.format(token.head, token.dep_, token)) process -(aux)-> Can process -(nsubj)-> you process -(ROOT)-> process text -(det)-> this process -(dobj)-> text process -(punct)-> ? 8 / 17
- 9. Get things done Named entity recognition: doc = nlp('The current capital of Japan is Tokyo.') print(doc.ents) (Japan, Tokyo) 9 / 17
- 10. What's next? More about spaCy Natural Language Processing in 10 Lines of Code How spaCy Works Incorporate with Deep learning library Deep Learning with custom pipelines and Keras Sense2vec with spaCy and Gensim 10 / 17
- 11. Try spaCy on the website Dependency parsing Named entity recognition Sentence similarity 11 / 17
- 12. textacy higher-level NLP built on spaCy Documentation / GitHub / API Reference textacy is a Python library for performing higher-level natural la nguage processing (NLP) tasks, built on the high-performance s paCy library. textacy focuses on tasks facilitated by the ready availability of t okenized, POS-tagged, and parsed text. 12 / 17
- 14. うちの研究室的に便利そうな機能 textacy.preprocess.preprocess_text x "broken" unicode replace all URL strings with 'URL' replace all email strings with 'EMAIL' replace all phone number strings with 'PHONE' replace all number-like strings with 'NUMBER' ... 14 / 17
- 15. うちの研究室的に便利そうな機能 textacy.preprocess.preprocess_text from textacy.preprocess import preprocess_text text = 'ここの研究室すごいよ！！！ http://www.cl.ecei.tohoku.ac.jp' preprocess_text(text, no_urls=True) 'ここの研究室すごいよ！！！ *URL*' 15 / 17
- 16. うちの研究室的に便利そうな機能 textacy.extract.pos_regex_matches from textacy.extract import pos_regex_matches from textacy.constants import POS_REGEX_PATTERNS list(pos_regex_matches(nlp('Can you process this text?'), POS_REGEX_PATTERNS['en']['NP'])) [this text] 16 / 17
- 17. ... and more! 17 / 17