CS571:: Part of-Speech Tagging
2 likes740 views
The document discusses part-of-speech tagging through supervised natural language processing. It involves collecting labeled text examples, extracting features from the text, training statistical models on the labeled examples and features, and evaluating the trained models. The document provides details on the feature extraction process, which involves identifying features from neighboring words, filtering low frequency features, assigning unique IDs to labels and features, and representing the data in a sparse column format for training softmax regression models to predict part-of-speech tags.
CS571:: Part of-Speech Tagging
- 1. Part-of-Speech Tagging Natural Language Processing Emory University Jinho D. Choi
- 2. Part-of-Speech Tagging 2 Classify the part-of-speech tag of each token. Jinho is a professor noun verb det. noun proper common3rd, present Supervised NLP 1. Collect 2. Train 3. Evaluate a. Design a processing algorithm. b. Extract (label, features) pairs. c. Vectorize labels and features. d. Build statistical models. https://github.com/emory-courses/cs571/wiki/Part-of-Speech-Tags
- 3. Feature Extraction 3 {wi, wi-1, wi+1, pi-1} Label F0 F1 F2 F3 NNP John ∅ is ∅ VBZ is John a NNP DT a is teacher VBZ NN teacher a ∅ DT John/NNP is/VBZ a/DT teacher/NN NNP John ∅ was ∅ VBD was John a NNP DT a was student VBD NN student a ∅ DT John/NNP was/VBD a/DT student/NN Extract the label and the features given the current state.
- 4. Feature Extraction 4 Label F0 F1 F2 F3 NNP John ∅ is ∅ VBZ is John a NNP DT a is teacher VBZ NN teacher a ∅ DT NNP John ∅ was ∅ VBD was John a NNP DT a was student VBD NN student a ∅ DT Filter out ones whose frequencies ≤ cutoff. Label {NNP:2, VBZ:1, DT:2, NN:2, VBD:1} F0 {John:2, is:1, a:2, teacher:1, was:1, student:1} F1 {John:2, is:1, a:2, was:1} F2 {is:1, a:2, teacher:1, was:1, student:1} F3 {NNP:2, VBZ:1, DT:2, VBD:1} cutoff = 1 Count
- 5. Feature Extraction 5 Assign an unique ID to each label and feature. Label {NNP:0, DT:1, NN:2} F0 {John:1, a:2} F1 {John:3, a:4} F2 {a:5} F3 {NNP:6, DT:7} Label F0 F1 F2 F3 NNP John ∅ is ∅ VBZ is John a NNP DT a is teacher VBZ NN teacher a ∅ DT NNP John ∅ was ∅ VBD was John a NNP DT a was student VBD NN student a ∅ DT 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 1 0 0 1 0 1 2 3 4 5 6 7 0 1 2 0 1 2
- 6. Softmax Regression 6 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0x p(y|X) = 1 Z(x) exp ( y + X 8k y,k · xk ) ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ƛNN ƛIN ƛVB ƛRB ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ? 1 b b b b p(y|X) = 1 Z(x) exp ( X 8k y,k · xk )
- 7. Column vs. Row 7 b f1 f2 … fd b f1 f2 … fd b f1 f2 … fd b f1 f2 … fd ƛNN ƛIN ƛVB ƛRB f1 NN VB IN RB NN VB IN RB NN VB IN RB f2 fd …vs. 1 0 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0x = f0 f9 f11 f23 f32 What is faster?
- 8. Column vs. Row 8 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 1 0 0 1 1 1 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1 0 0 0 1 0 0 1 0 1 2 3 4 5 6 7 0 1 2 0 1 2 00 01 02 10 11 12 20 21 22 30 31 32 40 41 42 50 51 52 60 61 62 70 71 72 Machine learning algorithm Y X Why group labels by features?
- 9. Ambiguity Classes 9 John ← NNP:100 study ←VB: 50, NN:50 interest ← JJ:70, NN:30 The likely part-of-speech tag. NNP VB_NN JJ or JJ_NN Collect the ambiguity classes before training. Use them as extra features.