Corpus Bootstrapping with NLTK
14 likes17,209 views
This document discusses using bootstrapping techniques to automatically create training corpora when manually annotated data is not available or too expensive. It describes translating an existing English sentiment corpus to Spanish as an example. The process involves translating the English examples, training an initial classifier, classifying new Spanish examples to build a corpus, manually correcting errors, retraining the classifier, and repeating the process with a lowered classification threshold. Similar techniques are outlined for bootstrapping a phrase extractor, including starting with a part-of-speech tagged corpus, annotating phrases, training taggers and chunkers, correcting errors, and adding to the corpus through iteration.
![Phrase Annotation
Hello world, [this is an important phrase].](https://image.slidesharecdn.com/corpusbootstrappingnltkindustrial-120228154407-phpapp02/85/Corpus-Bootstrapping-with-NLTK-23-320.jpg)
![Chunked & Tagged Phrase
Hello/N world/N ,/, [ this/DET is/V an/DET
important/ADJ phrase/N ] ./.](https://image.slidesharecdn.com/corpusbootstrappingnltkindustrial-120228154407-phpapp02/85/Corpus-Bootstrapping-with-NLTK-25-320.jpg)
![Extracting Phrases
import collections, nltk.data
from nltk import tokenize
from nltk.tag import untag
tagger = nltk.data.load('taggers/my_corpus_tagger.pickle')
chunker = nltk.data.load('chunkers/my_corpus_chunker.pickle')
def extract_phrases(t):
d = collections.defaultdict(list)
for sub in t.subtrees(lambda s: s.node != 'S'):
d[sub.node].append(' '.join(untag(sub.leaves())))
return d
sents = tokenize.sent_tokenize(text)
words = tokenize.word_tokenize(sents[0])
d = extract_phrases(chunker.parse(tagger.tag(words)))
# defaultdict(<type 'list'>, {'PHRASE_TAG': ['phrase']})](https://image.slidesharecdn.com/corpusbootstrappingnltkindustrial-120228154407-phpapp02/85/Corpus-Bootstrapping-with-NLTK-29-320.jpg)
Corpus Bootstrapping with NLTK
- 1. Corpus Bootstrapping with NLTK by Jacob Perkins
- 2. Jacob Perkins http://www.weotta.com http://streamhacker.com http://text-processing.com https://github.com/japerk/nltk-trainer @japerk
- 3. Problem you want to do NLProc many proven supervised training algorithms but you don’t have a training corpus
- 4. Solution make a custom training corpus
- 5. Problems with Manual Annotation takes time requires expertise expert time costs $$$
- 6. Solution: Bootstrap less time less expertise costs less requires thinking & creativity
- 7. Corpus Bootstrapping at Weotta review sentiment keyword classification phrase extraction & classification
- 8. Bootstrapping Examples english -> spanish sentiment phrase extraction
- 9. Translating Sentiment start with english sentiment corpus & classifier english -> spanish -> spanish
- 10. English -> Spanish -> Spanish 1. translate english examples to spanish 2. train classifier 3. classify spanish text into new corpus 4. correct new corpus 5. retrain classifier 6. add to corpus & goto 4 until done
- 11. Translate Corpus $ translate_corpus.py movie_reviews --source english --target spanish
- 12. Train Initial Classifier $ train_classifier.py spanish_movie_reviews
- 13. Create New Corpus $ classify_to_corpus.py spanish_sentiment --input spanish_examples.txt --classifier spanish_movie_reviews_NaiveBayes.pickle
- 14. Manual Correction 1. scan each file 2. move incorrect examples to correct file
- 15. Train New Classifier $ train_classifier.py spanish_sentiment
- 16. Adding to the Corpus start with >90% probability retrain carefully decrease probability threshold
- 17. Add more at a Lower Threshold $ classify_to_corpus.py categorized_corpus -- classifier categorized_corpus_NaiveBayes.pickle -- threshold 0.8 --input new_examples.txt
- 18. When are you done? what level of accuracy do you need? does your corpus reflect real text? how much time do you have?
- 19. Tips garbage in, garbage out correct bad data clean & scrub text experiment with train_classifier.py options create custom features
- 20. Bootstrapping a Phrase Extractor 1. find a pos tagged corpus 2. annotate raw text 3. train pos tagger 4. create pos tagged & chunked corpus 5. tag unknown words 6. train pos tagger & chunker 7. correct errors 8. add to corpus, goto 5 until done
- 21. NLTK Tagged Corpora English: brown, conll2000, treebank Portuguese: mac_morpho, floresta Spanish: cess_esp, conll2002 Catalan: cess_cat Dutch: alpino, conll2002 Indian Languages: indian Chinese: sinica_treebank see http://text-processing.com/demo/tag/
- 22. Train Tagger $ train_tagger.py treebank --simplify_tags
- 23. Phrase Annotation Hello world, [this is an important phrase].
- 24. Tag Phrases $ tag_phrases.py my_corpus --tagger treebank_simplify_tags.pickle --input my_phrases.txt
- 25. Chunked & Tagged Phrase Hello/N world/N ,/, [ this/DET is/V an/DET important/ADJ phrase/N ] ./.
- 26. Correct Unknown Words 1. find -NONE- tagged words 2. fix tags
- 27. Train New Tagger $ train_tagger.py my_corpus --reader nltk.corpus.reader.ChunkedCorpusReader
- 28. Train Chunker $ train_chunker.py my_corpus --reader nltk.corpus.reader.ChunkedCorpusReader
- 29. Extracting Phrases import collections, nltk.data from nltk import tokenize from nltk.tag import untag tagger = nltk.data.load('taggers/my_corpus_tagger.pickle') chunker = nltk.data.load('chunkers/my_corpus_chunker.pickle') def extract_phrases(t): d = collections.defaultdict(list) for sub in t.subtrees(lambda s: s.node != 'S'): d[sub.node].append(' '.join(untag(sub.leaves()))) return d sents = tokenize.sent_tokenize(text) words = tokenize.word_tokenize(sents[0]) d = extract_phrases(chunker.parse(tagger.tag(words))) # defaultdict(<type 'list'>, {'PHRASE_TAG': ['phrase']})
- 30. Final Tips error correction is faster than manual annotation find close enough corpora use nltk-trainer to experiment iterate -> quality no substitute for human judgement