Challenges of using Twitter for sentiment analysis
Download as PPT, PDF1 like4,902 views
The document discusses the significance of sentiment analysis on social media, emphasizing its impact on information processes and decision-making. It highlights traditional research methods such as experiments and interviews, as well as the increasing reliance on automated tools for sentiment tracking by businesses. The study also critiques the accuracy of these automated tools compared to manual analysis, pointing out challenges like contextual understanding and linguistic nuance.
![Canhoto 2015 8
Pratik Thakar, Head of creative content
for Coca-Cola Asia-Pacific:
“Every office has a listening centre
listening to what people are saying about
our brands, good and bad, 24 hours a
day. We look at what’s trending and how
we can respond [to discussions about
Coca-Cola] and to anything happening in
the world. (…) I believe that social media
is a big focus group. It’s a good way to
identify trends and what people are
talking about”
Source:
http://www.campaignasia.com/Article/402239,Dont+believe+
everything+you+hear+Cokes+Pratik+Thakar.aspx](https://image.slidesharecdn.com/cimm2015sentimentanalysis-150930115319-lva1-app6891/85/Challenges-of-using-Twitter-for-sentiment-analysis-8-320.jpg)
![As a consequence:
•Inaccurate representation of the overall sentiment
[towards coffee]
– Both sentiment polarity and emotional state
•Segments that should have been excluded from the
analysis were retained in the corpus of data
– Might skew results
•Concerns with the quality of the insights and
subsequent decisions
31Canhoto 2015](https://image.slidesharecdn.com/cimm2015sentimentanalysis-150930115319-lva1-app6891/85/Challenges-of-using-Twitter-for-sentiment-analysis-31-320.jpg)
![To improve accuracy [1/2]:
•Take into consideration the social context of the
conversation
– E.g., Tweets before or after the one being coded; wide
patterns (e.g., Mondays); cultural connotations (e.g., Japan
vs. UK)
– But what about sarcasm and highly contextualised uses of
language? (e.g., Sick)
32Canhoto 2015
Pratik Thakar:
“When people say good things, you don’t just take it as
it is. Someone might be asking them to say it; there
might be some design mechanism working. But when
people are unhappy, they go super-loud, and they are
genuine at that time. ”
Source:
http://www.campaignasia.com/Article/402239,Dont+believe+everything+you+hea
r+Cokes+Pratik+Thakar.aspx](https://image.slidesharecdn.com/cimm2015sentimentanalysis-150930115319-lva1-app6891/85/Challenges-of-using-Twitter-for-sentiment-analysis-32-320.jpg)
![To improve accuracy [2/2]:
•Develop dictionaries that reflect the specific syntax
and style
•Software solutions that “translate” commonly used
abbreviations and typos
– E.g., BRB – be right back
– Changing norms – e.g., LOL
•Familiarise with software
33Canhoto 2015](https://image.slidesharecdn.com/cimm2015sentimentanalysis-150930115319-lva1-app6891/85/Challenges-of-using-Twitter-for-sentiment-analysis-33-320.jpg)
Challenges of using Twitter for sentiment analysis
- 1. Studying sentiment on social media Ana Isabel Canhoto - Oxford Brookes University www.anacanhoto.com Canhoto 2015 1
- 2. 2 Emotions impact on: •Information retrieval •Information processing •Information retention •Decision-making •Behaviour •Assessment of consumption experiences Why study sentiment? Canhoto 2015 Image source: http://images.flatworldknowledge.com/sirgy/ sirgy-fig06_x003.jpg
- 3. 3 What are we talking about when we talk about sentiment analysis? More: http://www.mxmindia.com/2012/03/tweets- take-wing-in-airline-social-media-study/ Canhoto 2015
- 4. 4 Traditional approaches - Experiments More: http://www.psych.nyu.edu/amodiolab/Publi cations_files/Social_Psychological_Methods_ of_Emotion_Elicitation.pdf Canhoto 2015
- 5. 5 Traditional approaches – Interviews Canhoto 2015
- 6. Real time Unprompted No need to recall past behaviour Non-intrusive Cost-effective … 6 The Social Media Promise Canhoto 2015
- 8. Canhoto 2015 8 Pratik Thakar, Head of creative content for Coca-Cola Asia-Pacific: “Every office has a listening centre listening to what people are saying about our brands, good and bad, 24 hours a day. We look at what’s trending and how we can respond [to discussions about Coca-Cola] and to anything happening in the world. (…) I believe that social media is a big focus group. It’s a good way to identify trends and what people are talking about” Source: http://www.campaignasia.com/Article/402239,Dont+believe+ everything+you+hear+Cokes+Pratik+Thakar.aspx
- 9. 9 Many turning to third parties for automated tracking and analysis of SM conversations… Canhoto 2015 44% of businesses engaged in sentiment analysis Hilpern, K. 'In it to win it?' The Marketer, July-August 2012, pp.34-37 Estimated cumulative revenues cc $2bn in 2014 Source: http://breakthroughanalysis.com/2013/12/30/a w-re-aw-text-analytics-industry-study_start-ups- and-aquisition-activities_max-breitsprecher/ How accurate are these tools?
- 10. Promotional literature: accuracy rates of 70% - 80% (Davis & O’Flaherty, 2012) – Not clear how the coefficients were calculated – Not possible to independently verify these claims 10Canhoto 2015
- 11. 11 Click for updates Journal of MarketingManagement Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/rjmm20 ‘We (don’t)knowhowyou feel’ – a comparative study of automated vs. manual analysisof social media conversations Ana Isabel Canhoto a & Yuvraj Padmanabhan b a Faculty of Business, Oxford Brookes University, UK b Mindgraph, UK Published online: 18 Jun 2015. To cite thisarticle: Ana Isabel Canhoto & Yuvraj Padmanabhan (2015) ‘We (don’t) know how you feel’ – a comparative study of automated vs. manual analysis of social media conversations, Journal of Marketing Management, 31:9-10, 1141-1157, DOI: 10.1080/0267257X.2015.1047466 To link to this article: http://dx.doi.org/10.1080/0267257X.2015.1047466 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions Canhoto 2015 Open access
- 12. 12 Sources of vulnerability Canhoto 2015
- 13. • Accuracy: extent to which different researchers agree on the classification of a particular data object (Gwet, 2012) – System vs human coders – System A vs System B… 13Canhoto 2015
- 14. Conversations about coffee •Food and beverages = most widely discussed topic on social media (Forsyth, 2011) •‘Charged with a wide range of cultural meanings’ (Grinshpun, 2014) •Subject of many (netnographic) studies - e.g., Kozinets, 2002 14Canhoto 2015
- 15. • Sample of 200 tweets • Search terms: ‘coffee’ + variants ‘latte’, ‘mocha’, ‘cappuccino’, ‘espresso’ and ‘Americano’, as well as the terms ‘flavour’, ‘aroma’ and ‘caffeine’. • Multiple users – Exclude manufacturers and retailers. 15Canhoto 2015
- 16. Analysis - Stage 1: Polarity of emotion •Positive vs. negative – As per Koppel & Schler (2006): comments that did not express an emotion, were given the code ‘neutral’. •Manual + 2 automated tools 16Canhoto 2015
- 17. 17Canhoto 2015
- 18. Analysis - Stage 2: Type of emotion • As per Plutchik (2001) •Manual + 3 Automated tools 18Canhoto 2015
- 19. 19Canhoto 2015
- 20. 20Canhoto 2015
- 21. Messages where all types of coders agreed Examples: “Found a euro cent on my walk and have a great cup of coffee in hand. Monday is already off to a good start” “Feeling much more alive this morning now that I’ve had my coffee. Thank you #Nespresso”. Clearly positive! 21Canhoto 2015
- 22. Messages where automated tools agreed (but different from manual coding) Example: “In uni. I think without this cup of coffee I would hulk out” Very short segments 22Canhoto 2015
- 23. The rest Example: “The early shift sucks. Oh well at least my latte is yummy :) “ 23 Multiple objects Multiple emotions Canhoto 2015
- 24. Example: “100 copies of Ghosts sold overnight means a definite Starbucks run this morning. Possibly coffee out twice this week! Maybe even sushi!!” Lack of emotionally charged words 24Canhoto 2015
- 25. Example: “How the heck am I supposed to be able to sleep well without coffee in my system? fucking snow” Subtlety - Negative sentiment due to absence of product 25Canhoto 2015
- 26. Example: “Having coffee with my grandma before work right now. QT” Syntax and style, specially abbreviations and slang 26Canhoto 2015
- 27. Example: “This coffee shop needs to change there music up every once and a while. Or maybe I should go home” Target of emotion is not coffee! 27Canhoto 2015
- 28. 28Canhoto 2015
- 29. 29Canhoto 2015
- 30. 30 Compounded by: • Very short segments of text • Rich in abbreviations and slang • Typos or grammatical errors • Specific culture and netiquette of the media • Skills of data analystCanhoto 2015
- 31. As a consequence: •Inaccurate representation of the overall sentiment [towards coffee] – Both sentiment polarity and emotional state •Segments that should have been excluded from the analysis were retained in the corpus of data – Might skew results •Concerns with the quality of the insights and subsequent decisions 31Canhoto 2015
- 32. To improve accuracy [1/2]: •Take into consideration the social context of the conversation – E.g., Tweets before or after the one being coded; wide patterns (e.g., Mondays); cultural connotations (e.g., Japan vs. UK) – But what about sarcasm and highly contextualised uses of language? (e.g., Sick) 32Canhoto 2015 Pratik Thakar: “When people say good things, you don’t just take it as it is. Someone might be asking them to say it; there might be some design mechanism working. But when people are unhappy, they go super-loud, and they are genuine at that time. ” Source: http://www.campaignasia.com/Article/402239,Dont+believe+everything+you+hea r+Cokes+Pratik+Thakar.aspx
- 33. To improve accuracy [2/2]: •Develop dictionaries that reflect the specific syntax and style •Software solutions that “translate” commonly used abbreviations and typos – E.g., BRB – be right back – Changing norms – e.g., LOL •Familiarise with software 33Canhoto 2015
- 34. Studying sentiment on social media Ana Isabel Canhoto - Oxford Brookes University www.anacanhoto.com Canhoto 2015 34
Editor's Notes
- #2: Image: http://www.hirefuel.com/files/2013/06/social-media-chatter-300x219.jpg
- #13: Form: Syntax and style; Use of colloquialisms, abbreviations, symbols and emoticons Focus: Multiple sentiments and objects; Short text segments and use of non-textual elements Source: Subtlety; Use of irony and sarcasm Context: Contextual knowledge; Complexity of social media
- #35: Image: http://www.hirefuel.com/files/2013/06/social-media-chatter-300x219.jpg