Emotion Classification Using Massive Examples Extracted From The Web
- 1. Emotion Classification Using Massive Examples Extracted from the Web Ryoko TOKUHISA, Kentaro INUI, Yuji MATSUMOTO COLING’ 2008 Date: 2009-02-19
- 2. Outline Introduction Emotion Classification Experiments Conclusion
- 3. Introduction Goal: proposing a data-oriented method for inferring the emotion of a speaker conversing with a dialog system. Method Obtaining a huge collection of emotion-provoking event instances from the web. Decomposing the emotion classification task into two sub-steps: Coarse-grained: sentiment polarity classification . Fine-grained: emotion classification .
- 4. The Basic Idea Classification problem: a given input sentence is to be classified either into 10 emotion classes or neutral class . Basic idea: learning what emotion is typically provoked in what situation (emotion-provoking event). Ex.: “ I traveled for to get to the shop, but it was closed ” -> disappointing .
- 6. Building an EP Corpus Taking ten emotions ( happiness , fear …) as emotion classes. Building a handcrafted lexicon of emotion words (349 emotion words) classified into the ten emotions.
- 7. Building an EP Corpus cont. Using 349 emotion words to find sentences in the Web corpus that possibly contain emotion-provoking events. A subordinate clause was extracted as an emotion-provoking event instance if: It was subordinated to a matrix clause headed by an emotion word. The relation between the subordinate and matrix clauses is marked by one of the eight connectives ( ので , から , ため , て , のは , のが , ことは , ことが ). Ex.: “ I was disappointed that is suddenly started raining. ” the subordinate: it suddenly started raining . connective: that .
- 8. Building an EP Corpus cont. Apply above emotion lexicons and patterns to collection 1.3 million events. The evaluation of EP corpus by annotators.
- 9. Sentiment Polarity Classification Neutral sentences are not the majority in real Web texts. 1000 sentences randomly sampled from the web: Using the positive and negative examples stored in emotion-provoking corpus. Assuming the sentence to be neutral if the output of the model is near the decision boundary.
- 10. Sentiment Polarity Classification cont. SVMs and the features ( n -grams and the sentiment polarity of the word themselves). where, the sentiment dictionary (1880 positive words and 2490 negative words) from 50 thousand most frequent words sampled from the Web.
- 11. Emotion Classification Applying the KNN (k-nearest-neighbor) approach by using the EP corpus. Similarity measure: using cosine similarity between bag-of-words vectors ( I nstance and EP )
- 12. Experiment for Sentiment Classification Two test sets: TestSet1: 31 positive utterances, 34 negative utterances, and 25 neutral utterances. TestSet2: 1140 samples (judged Correct ) are 491 positives, 649 negatives sentences and additional 501 neutral sentences. Testing classification in both two-class and three-class setting. Metric: F-measure
- 14. Experiment for Emotion Classification Three test sets TestSet1 (2p, best) TestSet1 (1p, acceptable) TestSet2: using the results of their judgments on the correctness.
- 15. Baseline vs. KNN Baseline (Pointwise Mutual Information, PMI ) where e i ∈ { angry, disgust, fear, joy, sadness, surprise,… } cw j : each content word. Emotion class decision: KNN: 1-NN, 3-NN and 10-NN. One step: retrieve top-k examples from the EP corpus. Two step: retrieve top-k examples from the corresponding sentiment pool.
- 17. Conclusion Decomposing the emotion classification task into two sub-steps. Word n -gram features alone are more or less sufficient to classify sentence when a very large amount of training data is available. Two-step classification was effective for fine-grained emotion classification and outperform baseline model.