2. 2
Positive or negative movie review?
• unbelievably disappointing
• Full of zany characters and richly applied satire, and some
great plot twists
• this is the greatest screwball comedy ever filmed
• It was pathetic. The worst part about it was the boxing
scenes.
5. Twitter sentiment versus Gallup Poll of Consumer Confidence
Brendan O'Connor, Ramnath Balasubramanyan, Bryan R. Routledge, and Noah A. Smith. 2010.
From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series. In ICWSM-2010
6. 6
Twitter sentiment:
Johan Bollen, Huina Mao, Xiaojun Zeng. 2011.
Twitter mood predicts the stock market,
Journal of Computational Science 2:1, 1-8.
10.1016/j.jocs.2010.12.007.
8. 8
Target Sentiment on Twitter
• Twitter Sentiment App
• Alec Go, Richa Bhayani, Lei Huang. 2009.
Twitter Sentiment Classification using
Distant Supervision
9. 9
Sentiment analysis has many other names
• Opinion extraction
• Opinion mining
• Sentiment mining
• Subjectivity analysis
10. 10
Why sentiment analysis?
• Movie: is this review positive or negative?
• Products: what do people think about the new iPhone?
• Public sentiment: how is consumer confidence? Is despair
increasing?
• Politics: what do people think about this candidate or issue?
• Prediction: predict election outcomes or market trends from
sentiment
11. Scherer Typology of Affective States
• Emotion: brief organically synchronized … evaluation of a major event
• angry, sad, joyful, fearful, ashamed, proud, elated
• Mood: diffuse non-caused low-intensity long-duration change in subjective feeling
• cheerful, gloomy, irritable, listless, depressed, buoyant
• Interpersonal stances: affective stance toward another person in a specific interaction
• friendly, flirtatious, distant, cold, warm, supportive, contemptuous
• Attitudes: enduring, affectively colored beliefs, dispositions towards objects or persons
• liking, loving, hating, valuing, desiring
• Personality traits: stable personality dispositions and typical behavior tendencies
• nervous, anxious, reckless, morose, hostile, jealous
12. Scherer Typology of Affective States
• Emotion: brief organically synchronized … evaluation of a major event
• angry, sad, joyful, fearful, ashamed, proud, elated
• Mood: diffuse non-caused low-intensity long-duration change in subjective feeling
• cheerful, gloomy, irritable, listless, depressed, buoyant
• Interpersonal stances: affective stance toward another person in a specific interaction
• friendly, flirtatious, distant, cold, warm, supportive, contemptuous
• Attitudes: enduring, affectively colored beliefs, dispositions towards objects or persons
• liking, loving, hating, valuing, desiring
• Personality traits: stable personality dispositions and typical behavior tendencies
• nervous, anxious, reckless, morose, hostile, jealous
13. Sentiment Analysis
• Sentiment analysis is the detection of attitudes
“enduring, affectively colored beliefs, dispositions towards objects or persons”
1. Holder (source) of attitude
2. Target (aspect) of attitude
3. Type of attitude
• From a set of types
• Like, love, hate, value, desire, etc.
• Or (more commonly) simple weighted polarity:
• positive, negative, neutral, together with strength
4. Text containing the attitude
• Sentence or entire document
13
14. Sentiment Analysis
• Simplest task:
• Is the attitude of this text positive or negative?
• More complex:
• Rank the attitude of this text from 1 to 5
• Advanced:
• Detect the target, source, or complex attitude types
15. Sentiment Analysis
• Simplest task:
• Is the attitude of this text positive or negative?
• More complex:
• Rank the attitude of this text from 1 to 5
• Advanced:
• Detect the target, source, or complex attitude types
18. Sentiment Classification in Movie Reviews
• Polarity detection:
• Is an IMDB movie review positive or negative?
• Data: Polarity Data 2.0:
• http://www.cs.cornell.edu/people/pabo/movie-review-data
Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment
Classification using Machine Learning Techniques. EMNLP-2002, 79—86.
Bo Pang and Lillian Lee. 2004. A Sentimental Education: Sentiment Analysis Using
Subjectivity Summarization Based on Minimum Cuts. ACL, 271-278
19. IMDB data in the Pang and Lee database
when _star wars_ came out some twenty years ago
, the image of traveling throughout the stars has
become a commonplace image . […]
when han solo goes light speed , the stars change
to bright lines , going towards the viewer in lines
that converge at an invisible point .
cool .
_october sky_ offers a much simpler image–that of
a single white dot , traveling horizontally across the
night sky . [. . . ]
“ snake eyes ” is the most aggravating
kind of movie : the kind that shows so
much potential then becomes
unbelievably disappointing .
it’s not just because this is a brian
depalma film , and since he’s a great
director and one who’s films are always
greeted with at least some fanfare .
and it’s not even because this was a film
starring nicolas cage and since he gives a
brauvara performance , this film is hardly
worth his talents .
✓ ✗
20. Baseline Algorithm (adapted from Pang
and Lee)
• Tokenization
• Feature Extraction
• Classification using different classifiers
• Naïve Bayes
• MaxEnt
• SVM
21. 21
Sentiment Tokenization Issues
• Deal with HTML and XML markup
• Twitter mark-up (names, hash tags)
• Capitalization (preserve for
words in all caps)
• Phone numbers, dates
• Emoticons
• Useful code:
• Christopher Potts sentiment tokenizer
• Brendan O’Connor twitter tokenizer
[<>]? # optional hat/brow
[:;=8] # eyes
[-o*']? # optional nose
[)]([dDpP/:}{@|] # mouth
| #### reverse orientation
[)]([dDpP/:}{@|] # mouth
[-o*']? # optional nose
[:;=8] # eyes
[<>]? # optional hat/brow
Potts emoticons
22. 22
Extracting Features for Sentiment
Classification
• How to handle negation
• I didn’t like this movie
vs
• I really like this movie
• Which words to use?
• Only adjectives
• All words
• All words turns out to work better, at least on this data
23. Negation
Add NOT_ to every word between negation and following punctuation:
didn’t like this movie , but I
didn’t NOT_like NOT_this NOT_movie but I
Das, Sanjiv and Mike Chen. 2001. Yahoo! for Amazon: Extracting market sentiment from stock
message boards. In Proceedings of the Asia Pacific Finance Association Annual Conference (APFA).
Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment Classification
using Machine Learning Techniques. EMNLP-2002, 79—86.
25. 25
Binarized (Boolean feature) Multinomial Naïve Bayes
• Intuition:
• For sentiment (and probably for other text classification domains)
• Word occurrence may matter more than word frequency
• The occurrence of the word fantastic tells us a lot
• The fact that it occurs 5 times may not tell us much more.
• Boolean Multinomial Naïve Bayes
• Clips all the word counts in each document at 1
26. Boolean Multinomial Naïve Bayes: Learning
• Calculate P(cj) terms
• For each cj in C do
docsj all docs with class =cj
• Textj single doc containing all docsj
• Foreach word wk in Vocabulary
nk # of occurrences of wk in Textj
• From training corpus, extract Vocabulary
• Calculate P(wk | cj) terms
• Remove duplicates in each doc:
• For each word type w in docj
• Retain only a single instance of w
27. 27
Boolean Multinomial Naïve Bayes
on a test document d
• First remove all duplicate words from d
• Then compute NB using the same equation:
28. 28
Normal vs. Boolean Multinomial NB
Normal Doc Words Class
Training 1 Chinese Beijing Chinese c
2 Chinese Chinese Shanghai c
3 Chinese Macao c
4 Tokyo Japan Chinese j
Test 5 Chinese Chinese Chinese Tokyo Japan ?
Boolean Doc Words Class
Training 1 Chinese Beijing c
2 Chinese Shanghai c
3 Chinese Macao c
4 Tokyo Japan Chinese j
Test 5 Chinese Tokyo Japan ?
29. 29
Binarized (Boolean feature)
Multinomial Naïve Bayes
• Binary seems to work better than full word counts
• This is not the same as Multivariate Bernoulli Naïve Bayes
• MBNB doesn’t work well for sentiment or other text tasks
• Other possibility: log(freq(w))
B. Pang, L. Lee, and S. Vaithyanathan. 2002. Thumbs up? Sentiment Classification using Machine Learning
Techniques. EMNLP-2002, 79—86.
V. Metsis, I. Androutsopoulos, G. Paliouras. 2006. Spam Filtering with Naive Bayes – Which Naive Bayes?
CEAS 2006 - Third Conference on Email and Anti-Spam.
K.-M. Schneider. 2004. On word frequency information and negative evidence in Naive Bayes text
classification. ICANLP, 474-485.
JD Rennie, L Shih, J Teevan. 2003. Tackling the poor assumptions of naive bayes text classifiers. ICML 2003
30. Cross-Validation
• Break up data into 10 folds
• (Equal positive and negative
inside each fold?)
• For each fold
• Choose the fold as a
temporary test set
• Train on 9 folds, compute
performance on the test fold
• Report average
performance of the 10 runs
31. 31
Other issues in Classification
• MaxEnt and SVM tend to do better than Naïve Bayes
32. Problems:
What makes reviews hard to classify?
• Subtlety:
• Perfume review in Perfumes: the Guide:
• “If you are reading this because it is your darling fragrance,
please wear it at home exclusively, and tape the windows
shut.”
• Dorothy Parker on Katherine Hepburn
• “She runs the gamut of emotions from A to B”
32
33. 33
Thwarted Expectations
and Ordering Effects
• “This film should be brilliant. It sounds like a great plot,
the actors are first grade, and the supporting cast is
good as well, and Stallone is attempting to deliver a
good performance. However, it can’t hold up.”
• Well as usual Keanu Reeves is nothing special, but
surprisingly, the very talented Laurence Fishbourne is
not so good either, I was surprised.
![IMDB data in the Pang and Lee database
when _star wars_ came out some twenty years ago
, the image of traveling throughout the stars has
become a commonplace image . […]
when han solo goes light speed , the stars change
to bright lines , going towards the viewer in lines
that converge at an invisible point .
cool .
_october sky_ offers a much simpler image–that of
a single white dot , traveling horizontally across the
night sky . [. . . ]
“ snake eyes ” is the most aggravating
kind of movie : the kind that shows so
much potential then becomes
unbelievably disappointing .
it’s not just because this is a brian
depalma film , and since he’s a great
director and one who’s films are always
greeted with at least some fanfare .
and it’s not even because this was a film
starring nicolas cage and since he gives a
brauvara performance , this film is hardly
worth his talents .
✓ ✗](https://image.slidesharecdn.com/7sent-250620223249-ec06abfa/85/Sediment-analysis-what-is-Sediment-analysis-19-320.jpg)
![21
Sentiment Tokenization Issues
• Deal with HTML and XML markup
• Twitter mark-up (names, hash tags)
• Capitalization (preserve for
words in all caps)
• Phone numbers, dates
• Emoticons
• Useful code:
• Christopher Potts sentiment tokenizer
• Brendan O’Connor twitter tokenizer
[<>]? # optional hat/brow
[:;=8] # eyes
[-o*']? # optional nose
[)]([dDpP/:}{@|] # mouth
| #### reverse orientation
[)]([dDpP/:}{@|] # mouth
[-o*']? # optional nose
[:;=8] # eyes
[<>]? # optional hat/brow
Potts emoticons](https://image.slidesharecdn.com/7sent-250620223249-ec06abfa/85/Sediment-analysis-what-is-Sediment-analysis-21-320.jpg)
