Industrial strength - Natural Language Processing
Download as PPTX, PDF2 likes474 views
Jeffrey Williams presented on using the spaCy library for natural language processing. He began with an overview of key NLP concepts like tokenization, lemmatization, named entity recognition and part-of-speech tagging. He then demonstrated spaCy's features for these tasks and how to visualize outputs. Williams also showed how to apply spaCy to problems by extending its pipeline and training custom models. Finally, he reviewed alternatives to spaCy like NLTK, CoreNLP and cloud-based services from Microsoft and Google.
![Named Entity Extension
Adding Additional Entity Types
Need a few hundred labeled sentences
for a good start, mixin examples of other
entity types
Actual training is performed by looping
over the examples, makes a prediction
against golden parsed data
train_data = [
("Uber blew through $1 million a week", [(0, 4, 'ORG')]),
("Android Pay expands to Canada", [(0, 11, 'PRODUCT'), (23, 30,
'GPE')]),
("Spotify steps up Asia expansion", [(0, 8, "ORG"), (17, 21, "LOC")]),
("Google Maps launches location sharing", [(0, 11, "PRODUCT")]),
("Google rebrands its business apps", [(0, 6, "ORG")]),
("look what i found on google! 😂", [(21, 27, "PRODUCT")])]
Update a pre-trained Model
Need to provide many examples to meaningfully
improve the system — a few hundred
https://spacy.io/usage/training#section-ner](https://image.slidesharecdn.com/uadsugv6qfqq3gqjf5qd-signature-884d7fcdf8d2dc8dc8a2420311e0f9f5729e2a61049ccfc69c47c30e1d1f5707-poli-180611162257/85/Industrial-strength-Natural-Language-Processing-20-320.jpg)
Industrial strength - Natural Language Processing
- 1. Industrial Strength Natural Language Processing I am Jeffrey Williams I am here to provide meaning to unstructured text I work @ Label Insight You can find me at @jeffxor Label Insight is Hiring! https://www.labelinsight.com/careers/topic/engineering
- 2. Caveats ◇ I am not a linguist specialist ◇ I am not a natural language specialist ◇ I am not a data scientist ◇ I am a software engineer This talk is aimed at software engineers trying to tackle text problems by extract meaning or understanding
- 3. Agenda ◇ Natural Language Processing Concepts ◇ spacy.io Introduction ◇ Visualizations ◇ Applying spacy.io ◇ spacy.io Extensions ◇ Lessons Learnt ◇ Alternatives to spaCy.io
- 4. Let’s review some NLP concepts Sentence Boundary Detection Sentence boundaries are often marked by periods or other punctuation marks, but these same characters can serve other purposes Tokens/Word Segmentation Separate a chunk of continuous text into separate words. Text segmentation is a significant task requiring knowledge of the vocabulary and morphology. Stemming/Lemmatization reduce inflectional forms of a word to a common base form am, are, is -> be car, cars, car's, cars' -> car Named Entity Recognition Given a stream of text, determine which items in the text map to proper names, such as people or places, and what the type of each such name is (e.g. person, location, organization). Parts of Speech Tagging Given a sentence, determine the part of speech for each word. Many words, especially common ones, can serve as multiple parts of speech. Word sense disambiguation Many words have more than one meaning; we have to select the meaning which makes the most sense in context.
- 5. spaCy.io Introduction ◇ Open-source library for advanced (NLP) in Python ◇ Opinionated NLP library (not an API/Service) ◇ Number of pretrained models for common languages ◇ Great documentation and example code ◇ Helps build information extraction & natural language understanding systems spaCy.io is very powerful library that has many extension points allowing for training and pipeline configuration
- 6. spaCy.io Features Lemmatization Assigning the base forms of words. For example, the lemma of "was" is "be", and the lemma of "rats" is "rat". Rule-based Matching Finding sequences of tokens based on their texts and linguistic annotations, similar to regular expressions. Similarity Comparing words, text spans and documents and how similar they are to each other. (POS) Part-of-speech Tagging Assigning word types to tokens, like verb or noun. (NER) Named Entity Recognition Labelling named "real-world" objects, like persons, companies or locations. Dependency Parsing Assigning syntactic dependency labels, describing the relations between individual tokens, like subject or object.
- 7. Place your screenshot here Language Support spaCy v2.0 features new neural models for tagging, parsing and entity recognition. The models have been designed and implemented from scratch specifically for spaCy, to give you an unmatched balance of speed, size and accuracy. Combination of language (english), training data (web, news, etc), size of model (sm, md, lg) https://spacy.io/usage/models
- 8. Place your screenshot here Provided Named Entities From my experience with Locations it is not as well trained as Google Cloud Natural Language https://spacy.io/api/annotation#section-named-entities
- 9. Place your screenshot here Parts-of-Speech Tagging Maps all language-specific part-of-speech tags to a small, fixed set of word type tags following the Universal Dependencies scheme. https://spacy.io/api/annotation#section-pos-tagging
- 10. Visualizations Super simple and super powerful for development iteration
- 11. Place your screenshot here import spacy from spacy import displacy nlp = spacy.load('en') doc = nlp(u'This is a sentence.') displacy.serve(doc, style='dep') Dependency Visualization
- 12. Place your screenshot here import spacy from spacy import displacy text = """But Google is starting from behind. The company made a late push into hardware, and Apple’s Siri, available on iPhones, and Amazon’s Alexa software, which runs on its Echo and Dot devices, have clear leads in consumer adoption.""" nlp = spacy.load('custom_ner_model') doc = nlp(text) displacy.serve(doc, style='ent') Named Entity Visualization
- 13. spaCy.io Code Examples Examples of using applying spaCy.io’s building blocks to solve a problem
- 14. Navigating Parse Trees ◇ navigate the parse tree including subtrees attached to a word ◇ Noun chunks (noun plus the words describing the noun) ◇ terms head and child to describe the words connected by a single arc ◇ term dep is used for the arc label, ( type of syntactic relation)
- 15. Phrase Matcher ◇ efficiently match large terminology lists ◇ match sequences based on lists of token descriptions ◇ accepts match patterns in the form of Doc objects
- 16. spaCy.io Applied to the Real World Walk through applying to a new problem domain
- 17. Training Data Provide additional data to either adjust and existing model or build your own model. https://prodi.gy/ spaCy.io Extensions Functionality Number of extension points to add customizations ◇ Adjust pipeline ◇ Add new pipeline features ◇ Add functionality to core components ◇ Add callback functions into pipeline processes
- 18. spaCy.io Pipeline Disabling/Modifying If you don't need a particular component of the pipeline – for example, the tagger or the parser, you can disable loading it. Can sometimes make a big difference and improve loading speed. Custom Components Custom components can be added to the pipeline Allows for adding it before or after, tell spaCy to add it first or last in the pipeline, or define a custom name. Eg. add spell checking (hunspell) Extension Attributes allows you to set any custom attributes and methods on the Doc, Span and Token additional information relevant to your application, add new features and functionality to spaCy, and implement your own models Eg. improve spaCy's sentence boundary detectionhttps://spacy.io/usage/processing-pipelines
- 19. Place your screenshot here Processing Pipeline The Language object coordinates these components. It takes raw text and sends it through the pipeline, returning an annotated document. It also orchestrates training and serialization. https://spacy.io/usage/processing-pipelines
- 20. Named Entity Extension Adding Additional Entity Types Need a few hundred labeled sentences for a good start, mixin examples of other entity types Actual training is performed by looping over the examples, makes a prediction against golden parsed data train_data = [ ("Uber blew through $1 million a week", [(0, 4, 'ORG')]), ("Android Pay expands to Canada", [(0, 11, 'PRODUCT'), (23, 30, 'GPE')]), ("Spotify steps up Asia expansion", [(0, 8, "ORG"), (17, 21, "LOC")]), ("Google Maps launches location sharing", [(0, 11, "PRODUCT")]), ("Google rebrands its business apps", [(0, 6, "ORG")]), ("look what i found on google! 😂", [(21, 27, "PRODUCT")])] Update a pre-trained Model Need to provide many examples to meaningfully improve the system — a few hundred https://spacy.io/usage/training#section-ner
- 21. Place your screenshot here Custom Semantics ◇ Can be used to be trained to predict any type of tree structure over your input text ◇ Can be useful to for conversational applications, ◇ Train spaCy's parser to label intents and their targets, like attributes, quality, time and locations https://spacy.io/usage/training#section-tagger-parser
- 22. Attempt to summarize my learning curve both from implementation as well as business buyin spaCy.io Lessons Learnt
- 23. Start Simple! Define you key outcomes Visualize the data Experiment, iteration is key!
- 24. Educate Engage you SMEs Visualizations always help Opt for easy/understandable
- 25. Measurement System Metric Operations Metric Overall Business Metric
- 26. spaCy.io Alternatives There are many alternatives available they tend to fall into two categories, alternative libraries and hosted solutions
- 27. ◇ NLTK Natural Language Toolkit (Python) ◇ Stanford CoreNLP (Java) ◇ NLP4J (Java) Libraries allow you to configure, extend and train for your problem domain Alternate Libraries
- 28. ◇ Microsoft Azure Text Analytics ◇ Google Cloud Natural Language Hosted solutions provide a generic solution ◇ Well trained models ◇ Basic/Generic Named Entities ◇ Unable to model/train for your domain (yet!) Alternate Hosted Solutions
- 29. Thanks! Any questions? You can find me at: ◇ @jeffxor ◇ jwilliams@labelinsight.com ◇ https://speakerrate.com/speakers/181771 (Feedback) Label Insight is Hiring! https://www.labelinsight.com/careers/topic/engineering
- 30. Useful Information This presentation used the following resources: ◇ spacy.io ◇ spacy.io github ◇ explosion.ai/demos/ ◇ Natural Language Processing Wikipedia ◇ Stanford CoreNLP ◇ Microsoft Azure Text Analytics ◇ Google Cloud Natural Language