Computing with Affective Lexicons: Computational Linguistics Tutorial with Dan Jurafsky
5 likes10,111 views
The document discusses the concept of computing affective meanings by leveraging linguistic and psychological frameworks to model sentiment, emotion, personality, mood, and attitudes. It outlines various applications of affective computing, including detecting emotions in different contexts and generating appropriate responses in interactive systems. Additionally, it presents different lexicons and methodologies for analyzing affective states and their implications in natural language processing.
![SentiWordNet
Stefano
Baccianella,
Andrea
Esuli,
and
Fabrizio
Sebastiani.
2010
SENTIWORDNET
3.0:
An
Enhanced Lexical Resource
for
Sentiment Analysis
and
Opinion
Mining.
LREC-‐2010
• Home
page:
http://sentiwordnet.isti.cnr.it/
• All
WordNet synsets automatically
annotated
for
degrees
of
positivity,
negativity,
and
neutrality/objectiveness
• [estimable(J,3)]
“may
be
computed
or
estimated”
Pos 0 Neg 0 Obj 1
• [estimable(J,1)]
“deserving
of
respect
or
high
regard”
Pos .75 Neg 0 Obj .25](https://image.slidesharecdn.com/meetup-160202003641-160202172339/85/Computing-with-Affective-Lexicons-Computational-Linguistics-Tutorial-with-Dan-Jurafsky-12-320.jpg)
![Perceptual
Strength
Norms
Connell
and
Lynott norms
30
Rather than ask participants to condense their estimations of
sensory experience into a single concreteness or
imageability rating, modality-specific norming asks people
to rate how strongly they experience a variety of concepts
using each perceptual modality in turn (i.e., auditory,
gustatory, haptic, olfactory or visual: Lynott & Connell,
2009, in prep.; see also Connell & Lynott, 2010; Louwerse
to 700 (concrete or high-imageability).
Design & Analysis We ran stepwise regression analyses
with either concreteness or imageability rating as the
dependent variable, and ratings of auditory, gustatory,
haptic, olfactory and visual strength as competing
predictors. For analysis of consistency across the scales,
each dependent variable was split at its midpoint before
Table 1: Sample words, used in Experiments 1 and 2, for which perceptual strength ratings [0-5] match or mismatch ratings
of concreteness and imageability [100-700].
Perceptual strength
Word Auditory Gustatory Haptic Olfactory Visual Concreteness Imageability
soap 0.35 1.29 4.12 4.00 4.06 589 600
noisy 4.95 0.05 0.29 0.05 1.67 293 138
atom 1.00 0.63 0.94 0.50 1.38 481 499
republic 0.53 0.67 0.27 0.07 1.79 376 356
1429
Microsoft Excel
Worksheet](https://image.slidesharecdn.com/meetup-160202003641-160202172339/85/Computing-with-Affective-Lexicons-Computational-Linguistics-Tutorial-with-Dan-Jurafsky-30-320.jpg)
![Log
odds
ratio
Log
likelihood
ratio:
does
“horrible”
occur
more
%
in
corpus
A
or
B?
log 𝑃<
“ℎ𝑜𝑟𝑟𝑖𝑏𝑙𝑒” − log 𝑃D
(“ℎ𝑜𝑟𝑟𝑖𝑏𝑙𝑒”)
=
𝑙 𝑜𝑔
EFGHI< “JFKK%LMN”
∑ EFGHI(#FKP)QRST
UV
W
− 𝑙𝑜𝑔
EFGHID “JFKK%LMN”
∑ EFGHI(#FKP)QRST
UV
X
Log
odds
ratio:
does
“horrible”
have
higher
odds
in
A
or
B?
• 𝑙𝑜𝑔
YRZV[W “RSSU]^_”
`W
a&
YRZV[W “RSSU]^_”
`W
− 𝑙𝑜𝑔
YRZV[X “RSSU]^_”
`X
a&
YRZV[X “RSSU]^_”
`X
• 𝑙𝑜𝑔
EFGHI< “JFKK%LMN”
b<&EFGHI< “JFKK%LMN”
− 𝑙𝑜𝑔
EFGHID “JFKK%LMN”
bD&EFGHID “JFKK%LMN”](https://image.slidesharecdn.com/meetup-160202003641-160202172339/85/Computing-with-Affective-Lexicons-Computational-Linguistics-Tutorial-with-Dan-Jurafsky-61-320.jpg)
![Dialog
act
features
• Clarification
questions
What?
Excuse
me?
• Laughter
[Beginning
of
turn]
[End
of
turn]
• Appreciations
Awesome!
That’s
amazing!
Oh,
great
• Sympathy
That
sounds
terrible!
That’s
awful!
That
sucks!](https://image.slidesharecdn.com/meetup-160202003641-160202172339/85/Computing-with-Affective-Lexicons-Computational-Linguistics-Tutorial-with-Dan-Jurafsky-77-320.jpg)
Computing with Affective Lexicons: Computational Linguistics Tutorial with Dan Jurafsky
- 1. Computing with Affective Lexicons Dan Jurafsky NLP Meetup Jan 21, 2016
- 2. Affective meaning • Drawing on literatures in • affective computing (Picard 95) • linguistic subjectivity (Wiebe and colleagues) • social psychology (Pennebaker and colleagues) • Can we model the lexical semantics relevant to: • sentiment • emotion • personality • mood • attitudes 2
- 3. Why compute affective meaning? • Detecting: • sentiment towards politicians, products, countries, ideas • frustration of callers to a help line • stress in drivers or pilots • depression and other medical conditions • confusion in students talking to e-‐tutors • emotions in novels (e.g., for studying groups that are feared over time) • Could we generate: • emotions or moods for literacy tutors in the children’s storybook domain • emotions or moods for computer games • personalities for dialogue systems to match the user
- 4. Connotation in the lexicon • Words have connotation as well as sense • Can we build lexical resources that represent these connotations? • And use them in these computational tasks? 4
- 5. Scherer’s typology of affective states Emotion: relatively brief episode of synchronized response of all or most organismic subsystems in response to the evaluation of an event as being of major significance angry, sad, joyful, fearful, ashamed, proud, desperate Mood: diffuse affect state …change in subjective feeling, of low intensity but relatively long duration, often without apparent cause cheerful, gloomy, irritable, listless, depressed, buoyant Interpersonal stance: affective stance taken toward another person in a specific interaction, coloring the interpersonal exchange distant, cold, warm, supportive, contemptuous Attitudes: relatively enduring, affectively colored beliefs, preferences predispositions towards objects or persons liking, loving, hating, valuing, desiring Personality traits: emotionally laden, stable personality dispositions and behavior tendencies, typical for a person nervous, anxious, reckless, morose, hostile, envious, jealous
- 6. Computing with Affective Lexicons Sentiment Lexicons
- 7. Scherer’s typology of affective states Emotion: relatively brief episode of synchronized response of all or most organismic subsystems in response to the evaluation of an event as being of major significance angry, sad, joyful, fearful, ashamed, proud, desperate Mood: diffuse affect state …change in subjective feeling, of low intensity but relatively long duration, often without apparent cause cheerful, gloomy, irritable, listless, depressed, buoyant Interpersonal stance: affective stance taken toward another person in a specific interaction, coloring the interpersonal exchange distant, cold, warm, supportive, contemptuous Attitudes: relatively enduring, affectively colored beliefs, preferences predispositions towards objects or persons liking, loving, hating, valuing, desiring Personality traits: emotionally laden, stable personality dispositions and behavior tendencies, typical for a person nervous, anxious, reckless, morose, hostile, envious, jealous
- 8. The General Inquirer • Home page: http://www.wjh.harvard.edu/~inquirer • List of Categories: http://www.wjh.harvard.edu/~inquirer/homecat.htm • Spreadsheet: http://www.wjh.harvard.edu/~inquirer/inquirerbasic.xls • Categories: • Positiv (1915 words) and Negativ (2291 words) • Strong vs Weak, Active vs Passive, Overstated versus Understated • Pleasure, Pain, Virtue, Vice, Motivation, Cognitive Orientation, etc • Free for Research Use Philip J. Stone, Dexter C Dunphy, Marshall S. Smith, Daniel M. Ogilvie. 1966. The General Inquirer: A Computer Approach to Content Analysis. MIT Press
- 9. LIWC (Linguistic Inquiry and Word Count) Pennebaker, J.W., Booth, R.J., & Francis, M.E. (2007). Linguistic Inquiry and Word Count: LIWC 2007. Austin, TX • Home page: http://www.liwc.net/ • 2300 words, >70 classes • Affective Processes • negative emotion (bad, weird, hate, problem, tough) • positive emotion (love, nice, sweet) • Cognitive Processes • Tentative (maybe, perhaps, guess), Inhibition (block, constraint) • Pronouns, Negation (no, never), Quantifiers (few, many) • $30 or $90 fee
- 10. MPQA Subjectivity Cues Lexicon • Home page: http://www.cs.pitt.edu/mpqa/subj_lexicon.html • 6885 words from 8221 lemmas • 2718 positive • 4912 negative • Each word annotated for intensity (strong, weak) • GNU GPL 10 Theresa Wilson, Janyce Wiebe, and Paul Hoffmann (2005). Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis. Proc. of HLT-EMNLP-2005. Riloff and Wiebe (2003). Learning extraction patterns for subjective expressions. EMNLP-2003.
- 11. Bing Liu Opinion Lexicon • Bing Liu's Page on Opinion Mining • http://www.cs.uic.edu/~liub/FBS/opinion-‐lexicon-‐English.rar • 6786 words • 2006 positive • 4783 negative 11 MinqingHu and Bing Liu. Mining and Summarizing Customer Reviews. ACM SIGKDD-‐2004.
- 12. SentiWordNet Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010 SENTIWORDNET 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining. LREC-‐2010 • Home page: http://sentiwordnet.isti.cnr.it/ • All WordNet synsets automatically annotated for degrees of positivity, negativity, and neutrality/objectiveness • [estimable(J,3)] “may be computed or estimated” Pos 0 Neg 0 Obj 1 • [estimable(J,1)] “deserving of respect or high regard” Pos .75 Neg 0 Obj .25
- 13. Computing with Affective Lexicons Sentiment Lexicons
- 14. Computing with Affective Lexicons Emotion
- 15. Scherer’s typology of affective states Emotion: relatively brief episode of synchronized response of all or most organismic subsystems in response to the evaluation of an event as being of major significance angry, sad, joyful, fearful, ashamed, proud, desperate Mood: diffuse affect state …change in subjective feeling, of low intensity but relatively long duration, often without apparent cause cheerful, gloomy, irritable, listless, depressed, buoyant Interpersonal stance: affective stance taken toward another person in a specific interaction, coloring the interpersonal exchange distant, cold, warm, supportive, contemptuous Attitudes: relatively enduring, affectively colored beliefs, preferences predispositions towards objects or persons liking, loving, hating, valuing, desiring Personality traits: emotionally laden, stable personality dispositions and behavior tendencies, typical for a person nervous, anxious, reckless, morose, hostile, envious, jealous
- 16. Two families of theories of emotion • Atomic basic emotions • A finite list of 6 or 8, from which others are generated • Dimensions of emotion • Valence (positive negative) • Arousal (strong, weak) • Control 16
- 17. Ekman’s 6 basic emotions: Surprise, happiness, anger, fear, disgust, sadness
- 18. Valence/Arousal Dimensions High arousal, low pleasure High arousal, high pleasure anger excitement Low arousal, low pleasure Low arousal, high pleasure sadness relaxationarousal valence
- 19. Atomic units vs. Dimensions Distinctive • Emotions are units. • Limited number of basic emotions. • Basic emotions are innate and universal Dimensional • Emotions are dimensions. • Limited # of labels but unlimited number of emotions. • Emotions are culturally learned. Adapted from Julia Braverman
- 20. One emotion lexicon from each paradigm! 1. 8 basic emotions: • NRC Word-‐Emotion Association Lexicon (Mohammad and Turney 2011) 2. Dimensions of valence/arousal/dominance • Warriner, A. B., Kuperman, V., and Brysbaert, M. (2013) • Both built using Amazon Mechanical Turk 20
- 21. Plutchick’s wheel of emotion 21 • 8 basic emotions • in four opposing pairs: • joy–sadness • anger–fear • trust–disgust • anticipation–surprise
- 22. NRC Word-‐Emotion Association Lexicon 22 Mohammad and Turney 2011 • 10,000 words chosen mainly from earlier lexicons • Labeled by Amazon Mechanical Turk • 5 Turkers per hit • Give Turkers an idea of the relevant sense of the word • Result: amazingly anger 0 amazingly anticipation 0 amazingly disgust 0 amazingly fear 0 amazingly joy 1 amazingly sadness 0 amazingly surprise 1 amazingly trust 0 amazingly negative 0 amazingly positive 1
- 23. The AMT Hit 23 …
- 24. Lexicon of valence, arousal, and dominance • Warriner, A. B., Kuperman, V., and Brysbaert, M. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods 45, 1191-‐1207. • Supplementary data: This work is licensed under a Creative Commons Attribution-‐ NonCommercial-‐NoDerivs 3.0 Unported License. • Ratings for 14,000 words for emotional dimensions: • valence (the pleasantness of the stimulus) • arousal (the intensity of emotion provoked by the stimulus) • dominance(the degree of control exerted by the stimulus) 24
- 25. Lexicon of valence, arousal, and dominance • valence (the pleasantness of the stimulus) 9: happy, pleased, satisfied, contented, hopeful 1: unhappy, annoyed, unsatisfied, melancholic, despaired, or bored • arousal (the intensity of emotion provoked by the stimulus) 9: stimulated, excited, frenzied, jittery, wide-‐awake, or aroused 1: relaxed, calm, sluggish, dull, sleepy, or unaroused; • dominance(the degree of control exerted by the stimulus) 9: in control, influential, important, dominant, autonomous, or controlling 1: controlled, influenced, cared-‐for, awed, submissive, or guided • Again produced by AMT 25
- 26. Lexicon of valence, arousal, and dominance: Examples Valence Arousal Dominance vacation 8.53 rampage 7.56 self 7.74 happy 8.47 tornado 7.45 incredible 7.74 whistle 5.7 zucchini 4.18 skillet 5.33 conscious 5.53 dressy 4.15 concur 5.29 torture 1.4 dull 1.67 earthquake 2.14 26
- 27. Computing with Affective Lexicons Other Useful Lexicons
- 28. Concreteness versus abstractness • The degree to which the concept denoted by a word refers to a perceptible entity. • Do concrete and abstract words differ in connotation? • Storage and retrieval? • Bilingual processing? • Relevant for embodied view of cognition (Barsalou 1999 inter alia) • Do concrete words activate brain regions involved in relevant perception • Brysbaert, M., Warriner, A. B., and Kuperman, V. (2014) Concreteness ratings for 40 thousand generally known English word lemmasBehavior Research Methods 46, 904-‐911. • Supplementary data: This work is licensed under a Creative Commons Attribution-‐NonCommercial-‐NoDerivs 3.0 Unported License. • 37,058 English words and 2,896 two-‐word expressions ( “zebra crossing” and “zoom in”), • Rating from 1 (abstract) to 5 (concrete) • Calibrator words: • shirt, infinity, gas, grasshopper, marriage, kick, polite, whistle, theory, and sugar 28
- 29. Concreteness versus abstractness • Brysbaert, M., Warriner, A. B., and Kuperman, V. (2014) Concreteness ratings for 40 thousand generally known English word lemmasBehavior Research Methods 46, 904-‐911. • Supplementary data: This work is licensed under a Creative Commons Attribution-‐NonCommercial-‐NoDerivs 3.0 Unported License. • Some example ratings from the final dataset of 40,000 words and phrases banana 5 bathrobe 5 bagel 5 brisk 2.5 badass 2.5 basically 1.32 belief 1.19 although 1.07 29
- 30. Perceptual Strength Norms Connell and Lynott norms 30 Rather than ask participants to condense their estimations of sensory experience into a single concreteness or imageability rating, modality-specific norming asks people to rate how strongly they experience a variety of concepts using each perceptual modality in turn (i.e., auditory, gustatory, haptic, olfactory or visual: Lynott & Connell, 2009, in prep.; see also Connell & Lynott, 2010; Louwerse to 700 (concrete or high-imageability). Design & Analysis We ran stepwise regression analyses with either concreteness or imageability rating as the dependent variable, and ratings of auditory, gustatory, haptic, olfactory and visual strength as competing predictors. For analysis of consistency across the scales, each dependent variable was split at its midpoint before Table 1: Sample words, used in Experiments 1 and 2, for which perceptual strength ratings [0-5] match or mismatch ratings of concreteness and imageability [100-700]. Perceptual strength Word Auditory Gustatory Haptic Olfactory Visual Concreteness Imageability soap 0.35 1.29 4.12 4.00 4.06 589 600 noisy 4.95 0.05 0.29 0.05 1.67 293 138 atom 1.00 0.63 0.94 0.50 1.38 481 499 republic 0.53 0.67 0.27 0.07 1.79 376 356 1429 Microsoft Excel Worksheet
- 31. Computing with Affective Lexicons Semi-‐supervised algorithms for learning affective lexicons
- 32. Semi-‐supervised learning of lexicons • Use a small amount of information • A few labeled examples • A few hand-‐built patterns • To bootstrap a lexicon 32
- 33. Hatzivassiloglou and McKeown intuition for identifying word polarity • Adjectives conjoined by “and” have same polarity • Fair and legitimate, corrupt and brutal • *fair and brutal, *corrupt and legitimate • Adjectives conjoined by “but” do not • fair but brutal 33 VasileiosHatzivassiloglou and Kathleen R. McKeown. 1997. Predicting the Semantic Orientation of Adjectives. ACL, 174–181
- 34. Hatzivassiloglou & McKeown 1997 Step 1 • Label seed set of 1336 adjectives (all >20 in 21 million word WSJ corpus) • 657 positive • adequate central clever famous intelligent remarkable reputed sensitive slender thriving… • 679 negative • contagious drunken ignorant lanky listless primitive strident troublesome unresolved unsuspecting… 34
- 35. Hatzivassiloglou & McKeown 1997 Step 2 • Expand seed set to conjoined adjectives 35 nice, helpful nice, classy
- 36. Hatzivassiloglou & McKeown 1997 Step 3 • Supervised classifier assigns “polarity similarity” to each word pair, resulting in graph: 36 classy nice helpful fair brutal irrationalcorrupt
- 37. Hatzivassiloglou & McKeown 1997 Step 4 • Clustering for partitioning the graph into two 37 classy nice helpful fair brutal irrationalcorrupt + -‐
- 38. Output polarity lexicon • Positive • bold decisive disturbing generous good honest important large mature patient peaceful positive proud sound stimulating straightforward strange talented vigorous witty… • Negative • ambiguous cautious cynical evasive harmful hypocritical inefficient insecure irrational irresponsible minor outspoken pleasant reckless risky selfish tedious unsupported vulnerable wasteful… 38
- 39. Output polarity lexicon • Positive • bold decisive disturbing generous good honest important large mature patient peaceful positive proud sound stimulating straightforward strange talented vigorous witty… • Negative • ambiguous cautious cynical evasive harmful hypocritical inefficient insecure irrational irresponsible minor outspoken pleasant reckless risky selfish tedious unsupported vulnerable wasteful… 39
- 40. Turney Algorithm 1. Extract a phrasal lexicon from reviews 2. Learn polarity of each phrase 3. Rate a review by the average polarity of its phrases 40 Turney (2002): Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
- 41. Extract two-‐word phrases with adjectives First Word Second Word Third Word (not extracted) JJ NN or NNS anything RB, RBR, RBS JJ Not NN nor NNS JJ JJ Not NN or NNS NN or NNS JJ Nor NN nor NNS RB, RBR, or RBS VB, VBD, VBN, VBG anything 41
- 42. How to measure polarity of a phrase? • Positive phrases co-‐occur more with “excellent” • Negative phrases co-‐occur more with “poor” • But how to measure co-‐occurrence? 42
- 43. Pointwise Mutual Information • Pointwise mutual information: • How much more do events x and y co-‐occur than if they were independent? • PMI between two words: • How much more do two words co-‐occur than if they were independent? PMI(word1,word2 ) = log2 P(word1,word2) P(word1)P(word2) PMI(X,Y) = log2 P(x,y) P(x)P(y)
- 44. How to Estimate Pointwise Mutual Information • Query search engine (Altavista) • P(word) estimated by hits(word)/N • P(word1,word2) by hits(word1 NEAR word2)/N • (More correctly the bigram denominator should be kN, because there are a total of N consecutive bigrams (word1,word2), but kN bigrams that are k words apart, but we just use N on the rest of this slide and the next.) PMI(word1,word2 ) = log2 1 N hits(word1 NEAR word2) 1 N hits(word1)1 N hits(word2)
- 45. Does phrase appear more with “poor” or “excellent”? 45 Polarity(phrase) = PMI(phrase,"excellent")− PMI(phrase,"poor") = log2 hits(phrase NEAR "excellent")hits("poor") hits(phrase NEAR "poor")hits("excellent") ! " # $ % & = log2 hits(phrase NEAR "excellent") hits(phrase)hits("excellent") hits(phrase)hits("poor") hits(phrase NEAR "poor") = log2 1 N hits(phrase NEAR "excellent") 1 N hits(phrase) 1 N hits("excellent") − log2 1 N hits(phrase NEAR "poor") 1 N hits(phrase) 1 N hits("poor")
- 46. Phrases from a thumbs-‐up review 46 Phrase POS tags Polarity online service JJ NN 2.8 online experience JJ NN 2.3 direct deposit JJ NN 1.3 local branch JJ NN 0.42 … low fees JJ NNS 0.33 true service JJ NN -0.73 other bank JJ NN -0.85 inconveniently located JJ NN -1.5 Average 0.32
- 47. Phrases from a thumbs-‐down review 47 Phrase POS tags Polarity direct deposits JJ NNS 5.8 online web JJ NN 1.9 very handy RB JJ 1.4 … virtual monopoly JJ NN -2.0 lesser evil RBR JJ -2.3 other problems JJ NNS -2.8 low funds JJ NNS -6.8 unethical practices JJ NNS -8.5 Average -1.2
- 48. Using WordNet to learn polarity • WordNet: online thesuarus • Create positive (“good”) and negative seed-‐words (“terrible”) • Find Synonyms and Antonyms • Positive Set: Add synonyms of positive words (“well”) and antonyms of negative words • Negative Set: Add synonyms of negative words (“awful”) and antonyms of positive words (”evil”) • Repeat, following chains of synonyms • Filter48 S.M. Kim and E. Hovy. 2004. Determining the sentiment of opinions. COLING 2004 M. Hu and B. Liu. Mining and summarizing customer reviews. In Proceedings of KDD, 2004
- 49. Summary on semi-‐supervised lexicon learning • Advantages: • Can be domain-‐specific • Can be more robust (more words) • Intuition • Start with a seed set of words (‘good’, ‘poor’) • Find other words that have similar polarity: • Using “and” and “but” • Using words that occur nearby in the same document • Using WordNet synonyms and antonyms
- 50. Computing with Affective Lexicons Supervised Learning of Sentiment Lexicons
- 51. Learn word sentiment supervised by online review scores • Review datasets • IMDB, Goodreads, Open Table, Amazon, Trip Advisor • Each review has a score (1-‐5, 1-‐10, etc) • Just count how many times each word occurs with each score • (and normalize) 51 Potts, Christopher. 2011. On the negativity of negation. SALT 20, 636-‐659. Potts 2011 NSF Workshop talk.
- 52. Analyzing the polarity of each word in IMDB • How likely is each word to appear in each sentiment class? • Count(“bad”) in 1-‐star, 2-‐star, 3-‐star, etc. • But can’t use raw counts: • Instead, likelihood: • Make them comparable between words • Scaled likelihood: Potts, Christopher. 2011. On the negativity of negation. SALT 20, 636-‐659. P(w | c) = f (w,c) f (w,c) w∈c ∑ P(w | c) P(w)
- 53. IMDB – 33,515 tokens Category -0.50 -0.39 -0.28 -0.17 -0.06 0.06 0.17 0.28 0.39 0.50 0.04 0.09 0.17 Cat = -0.13 (p = 0.284) Cat^2 = -5.37 (p < 0.001) IMDB – 176,264 tokens Category -0.50 -0.39 -0.28 -0.17 -0.06 0.06 0.17 0.28 0.39 0.50 0.05 0.09 0.13 Cat = -0.43 (p < 0.001) Cat^2 = -3.6 (p < 0.001) “Potts diagrams” Potts, Christopher. 2011. NSF workshop on restructuring adjectives. good great excellent disappointing bad terrible totally absolutely utterly somewhat fairly pretty Positive scalars Negative scalars Emphatics Attenuators
- 54. More complicated versions of likelihood: Log odds ratio informative Dirichlet prior Monroe, Colaresi and Quinn (2008) Find words that are statistically overrepresented in a particular category of review compared to another 𝛿" # (%&') = 𝑙𝑜𝑔 𝑦# % + 𝛼# 𝑛% + 𝛼2 − (𝑦# % + 𝛼#) − 𝑙 𝑜𝑔 𝑦# ' + 𝛼# 𝑛' + 𝛼2 − (𝑦# ' + 𝛼#) (𝑛% is the size of corpus i, 𝑛' is the size of corpus j, 𝑦# % is the count of word w in corpus i, 𝑦# ' is the count of word w in corpus j, 𝛼2 is the size of the background corpus, and 𝛼# is the count of word w in the background corpus.) 𝜎5 𝛿"# %&' ≈ 1 𝑦# % + 𝛼# + 1 𝑦# ' + 𝛼# • Final statistic for a word: z-‐score of its log-‐odds-‐ratio: 𝛿"# %&' 𝜎5 𝛿" # %&'
- 55. Or use regression coefficients to weight words • Train a classifier based on supervised data • Predict: human-‐labeled connotation of a document • From: all the words and bigrams in it • Use the regression coefficients as the weights 55
- 56. Computing with Affective Lexicons Using the lexicons to detect affect
- 57. Lexicons for detecting document affect: Simplest unsupervised method • Sentiment: • Sum the weights of each positive word in the document • Sum the weights of each negative word in the document • Choose whichever value (positive or negative) has higher sum • Emotion: • Do the same for each emotion lexicon 57
- 58. Lexicons for detecting document affect: Simplest supervised method • Build a classifier • Predict sentiment (or emotion, or personality) given features • Use “counts of lexicon categories” as a features • Sample features: • LIWC category “cognition” had count of 7 • NRC Emotion category “anticipation” had count of 2 • Baseline • Instead use counts of all the words and bigrams in the training set • This is hard to beat • But only works if the training and test sets are very similar58
- 59. Computing with Affective Lexicons Sample affective task: Cheap vs. Expensive Restaurants
- 60. Dan Jurafsky, Victor Chahuneau, Bryan R. Routledge, and Noah A. Smith. 2014. Narrative framing of consumer sentiment in online restaurant reviews. First Monday 19:4 Framing in restaurant reviews The bartender... absolutely horrible... we waited 10 min before we even got her attention... and then we had to wait 45 -‐ FORTY FIVE! -‐ minutes for our entrees… stalk the waitress to get the cheque… she didn't make eye contact or even break her stride to wait for a response 900,000 online reviews
- 61. Log odds ratio Log likelihood ratio: does “horrible” occur more % in corpus A or B? log 𝑃< “ℎ𝑜𝑟𝑟𝑖𝑏𝑙𝑒” − log 𝑃D (“ℎ𝑜𝑟𝑟𝑖𝑏𝑙𝑒”) = 𝑙 𝑜𝑔 EFGHI< “JFKK%LMN” ∑ EFGHI(#FKP)QRST UV W − 𝑙𝑜𝑔 EFGHID “JFKK%LMN” ∑ EFGHI(#FKP)QRST UV X Log odds ratio: does “horrible” have higher odds in A or B? • 𝑙𝑜𝑔 YRZV[W “RSSU]^_” `W a& YRZV[W “RSSU]^_” `W − 𝑙𝑜𝑔 YRZV[X “RSSU]^_” `X a& YRZV[X “RSSU]^_” `X • 𝑙𝑜𝑔 EFGHI< “JFKK%LMN” b<&EFGHI< “JFKK%LMN” − 𝑙𝑜𝑔 EFGHID “JFKK%LMN” bD&EFGHID “JFKK%LMN”
- 62. Log odds ratio with a prior Log odds ratio: 𝑙𝑜𝑔 cW “JFKK%LMN” bW&cW “JFKK%LMN” − 𝑙𝑜𝑔 cX “JFKK%LMN” bX&cX “JFKK%LMN” With a prior: 𝑙𝑜𝑔 cW “JFKK%LMN” dceSURS (“JFKK%LMN”) bWdbeSURS&(cW “JFKK%LMN” dceSURS(“JFKK%LMN”) − 𝑙𝑜𝑔 cX “JFKK%LMN” dceSURS (“JFKK%LMN”) bXdbeSURS&(cX “JFKK%LMN” dceSURS(“JFKK%LMN”)
- 63. Log odds ratio informative Dirichlet prior Monroe, Colaresi and Quinn (2008) Find words that are statistically overrepresented in a particular category of review compared to another 𝛿" # (%&') = 𝑙𝑜𝑔 𝑦# % + 𝛼# 𝑛% + 𝛼2 − (𝑦# % + 𝛼#) − 𝑙 𝑜𝑔 𝑦# ' + 𝛼# 𝑛' + 𝛼2 − (𝑦# ' + 𝛼#) (𝑛% is the size of corpus i, 𝑛' is the size of corpus j, 𝑦# % is the count of word w in corpus i, 𝑦# ' is the count of word w in corpus j, 𝛼2 is the size of the background corpus, and 𝛼# is the count of word w in the background corpus.) 𝜎5 𝛿"# %&' ≈ 1 𝑦# % + 𝛼# + 1 𝑦# ' + 𝛼# • Final statistic for a word: z-‐score of its log-‐odds-‐ratio: 𝛿"# %&' 𝜎5 𝛿" # %&'
- 64. Top 50 words associated with * reviews by Monroe, et al. (2008) method LinguisticClass Words in Class Negative sentiment worst, rude, terrible, horrible, bad, awful, disgusting, bland, tasteless, gross, mediocre, overpriced, worse, poor Linguistic negation no, not 1 pl pronouns we, us, our 3rd pronouns she, he, her, him Past tense verbs was, were, asked, told, said, did, charged, waited, left, took Narrative sequencers after, then Common nouns manager, waitress, waiter, customer, customers, attitude, waste, poisoning, money, bill, minutes Irrealis modals would, should
- 65. Language of bad reviews? • Negative sentiment language horrible awful terrible bad disgusting • Past narratives about people waited, didn’t, was he, she, his, her, manager, customer, waitress, waiter • Frequent mentions of we and us ... we were ignored until we flagged down one waiter to go get our waitress …
- 66. Other narratives with this language A genre using: Past tense, we/us, negative, people narratives Texts written by people suffering trauma • James Pennebakerlab • Blog posts after 9-‐11 • Student writings after campus tragedies • Use of “we”: seeking solace in community 1-‐star reviews are trauma narratives! • The lesson of reviews: It’s all about personal interaction
- 67. Positive Reviews? orgasmic pastry seductively seared... very naughty pork belly addicted to wings the fries are like crack ….crave… cupcakes
- 68. Why the drug metaphor? • Used for junk foods: chicken wings, french fries, pizza, chocolate • Assuage the guilt • It’s not my fault, I had no control, the cupcake made me eat it • Women more likely to use the metaphor of drugs than men • Consistent with other research that suggests women are more pressured to conform to healthy eating and mention food cravings (Rozin et al. 1991, inter alia)
- 69. Language and identity in menu language Online menus from 6562 restaurants
- 70. Lots of adjectives in $$ menus Crispy white-‐meat tenders served with a creamy Creole sauce rich, creamy spinach artichoke dip Creamy, homemade fettuccine alfredo zesty chili pepper cream sauce zesty, rich, golden brown, crispy, creamy
- 71. Lots of vague filler words in $ menus The delicious taste delicious outdoor grill flavor flavorful entrées flavorful ancho-‐chile two freshly made sides freshly steamed broccoli Delicious, freshly, flavorful
- 72. Computing with Affective Lexicons Interpersonal Stance Detection
- 73. Scherer’s typology of affective states Emotion: relatively brief episode of synchronized response of all or most organismic subsystems in response to the evaluation of an event as being of major significance angry, sad, joyful, fearful, ashamed, proud, desperate Mood: diffuse affect state …change in subjective feeling, of low intensity but relatively long duration, often without apparent cause cheerful, gloomy, irritable, listless, depressed, buoyant Interpersonal stance: affective stance taken toward another person in a specific interaction, coloring the interpersonal exchange distant, cold, warm, supportive, contemptuous Attitudes: relatively enduring, affectively colored beliefs, preferences predispositions towards objects or persons liking, loving, hating, valuing, desiring Personality traits: emotionally laden, stable personality dispositions and behavior tendencies, typical for a person nervous, anxious, reckless, morose, hostile, envious, jealous
- 74. Automatically Extracting Social Meaning from Speed Dates Rajesh Ranganath, Dan Jurafsky, and Daniel A. McFarland. 2013. Detecting friendly, flirtatious, awkward, and assertive speech in speed-‐dates. Computer Speech and Language 27:1, 89-‐115. McFarland, Daniel, Dan Jurafsky, and Craig M. Rawlings. 2013. "Making the Connection: Social Bonding in Courtship Situations.” American Journal of Sociology.
- 75. Affect extraction: of course it’s not just the lexicon • Detecting interpersonal stance in conversation • Speed dating study, 1000 4-‐minute speed dates • Subjects labeled selves and each other for • friendly (each on a scale of 1-‐10) • awkward • flirtatious • assertive 75 Ranganath et al (2013), McFarland et al (2014)
- 76. Linguistic features we examined • Words: • HEDGES: kind of, sort of, a little, I don’t know, I guess • NEGEMOTION: bad, weird, crazy, problem, tough, awkward, boring • LOVE: love, loved, loving, passion, passions, passionate • WORK: research, advisor, lab, work, finish, PhD, department • I: I, me, mine, my, myself, you, your, yours, etc. • Prosody • pitch ceiling, pitch floor, energy, rate of speech
- 77. Dialog act features • Clarification questions What? Excuse me? • Laughter [Beginning of turn] [End of turn] • Appreciations Awesome! That’s amazing! Oh, great • Sympathy That sounds terrible! That’s awful! That sucks!
- 78. Positive and negative assessments Sympathy (that’s|that is|that seems|it is|that sounds) (very|really|a little|sort of)? (terrible|awful|weird|sucks|a problem|tough|too bad) Appreciations (“Positive feedback”) (Oh)? (Awesome|Great|All right|Man|No kidding|wow|my god) That (‘s|is|sounds|would be) (so|really)? (great|funny|good|interesting|neat|amazing|nice|not bad|fun) (Goodwin, 1996; Goodwin and Goodwin, 1987; Jurafsky et al., 1998)
- 79. Interruptions A: Not necessarily. I mean it happens to not necessarily be my thing, but there are plenty of-‐-‐ B: No, no, I understand your point.
- 80. Results on affect extraction • Friendliness • -‐negative emotional words • -‐hedges (kind of, sort of, a little) • higher pitch • Awkwardness • +negation • +hedges • +questions 80
- 81. How does clicking happen? • Sociology literature: • bonding or “sense of connection” is caused by • homophily: select mate who shares your attributes and attitudes • motives and skills • mutual coordination and excitement • (Durkheim: religious rituals, unison singing, military) • What is the role of language? • Background: speed dating has power asymmetry • women are pickier • Lot of other asymmetric role relationships (teacher-‐ student, doctor-‐patient, boss-‐employee, etc.)
- 82. Our hypothesis: targeting of the empowered party • The conversational target is the woman • both parties should talk about her more • The woman’s face is important • the man should align to the woman and show understanding • The woman’s engagement is key • in a successful bonding, she should be engaged
- 83. Results: Clicking associated with: • both parties talk about the woman • women use I, • men use you • man supports woman’s face • men use appreciations and sympathy, • men accommodate women’s laughter • men interrupt with collaborative completions • woman is engaged • women raise their pitch, vary loudness and pitch • women avoid hedges Hierarchical regression dyad model, net of actor, partner, dyad features
- 84. Function of Interruptions? • Control: Used by men to commence topics (Zimmerman and West 1975; West 1985) • Shared meaning, alignment, engagement (Tannen 1994; Coates 1996, 1997), collaborative floor (Edelsky 1981).
- 85. We found: interruptions are joint construction (“collaborative completions”) • a turn where a speaker completes the utterance begun by the alter (Lerner, 1991; Lerner, 1996). So are you almost-‐-‐ On my way out, yeah.
- 86. Or showing shared understanding Female: I didn’t used to like it but now I’m— Male: Oh same for me.…
- 87. Computing with Affective Lexicons Interpersonal Stance Detection
- 88. Scherer’s typology of affective states Emotion: relatively brief episode of synchronized response of all or most organismic subsystems in response to the evaluation of an event as being of major significance angry, sad, joyful, fearful, ashamed, proud, desperate Mood: diffuse affect state …change in subjective feeling, of low intensity but relatively long duration, often without apparent cause cheerful, gloomy, irritable, listless, depressed, buoyant Interpersonal stance: affective stance taken toward another person in a specific interaction, coloring the interpersonal exchange distant, cold, warm, supportive, contemptuous Attitudes: relatively enduring, affectively colored beliefs, preferences predispositions towards objects or persons liking, loving, hating, valuing, desiring Personality traits: emotionally laden, stable personality dispositions and behavior tendencies, typical for a person nervous, anxious, reckless, morose, hostile, envious, jealous
- 89. Summary: Connotation in the lexicon • Words have various connotational aspects • Methods for building connotation lexicons Based on theoretical models of emotion, sentiment • By hand (mainly using crowdsourcing) • Semi-‐supervised learning from seed words • Fully supervised (when you can find a convenient signal in the world) • Applying lexicons to detect affect and sentiment • Unsupervised: pick simple majority sentiment (positive/negative words) • Supervised: learn weights for each lexical category • But hand-‐built phrases, and conversational features also helpful89