Open Analytics: Building Effective Frameworks for Social Media Analysis
Download as PPTX, PDF3 likes3,168 views
The document outlines an analytic framework for effectively analyzing social media data. It discusses common pitfalls to avoid, such as analyzing what is said rather than why. The framework involves capturing relevant social media data, reporting on it, and analyzing it to answer operational questions. A case study applies this by analyzing Twitter sentiment about a new video game. It finds hashtags are generic and expands the query to improve understanding attitudes toward the product. The document concludes by recommending segmentation, graph analysis, and tying metrics to actions.
Open Analytics: Building Effective Frameworks for Social Media Analysis
- 1. Open Analytics NYC – 11/08/2012 Building Effective Frameworks for Social Media Analysis
- 2. Agenda • Social Media: An Intelligence perspective • Common Analytic Pitfalls • An Analytic Framework • Case Study: Brand Management – Problem Definition – Source Selection – Data Capture – Data Reporting – Data Analysis • Ways Forward, Future Analysis • Questions?
- 3. Intelligence • Intelligence is information that has been transformed to meet an operational need Data Intelligence Operational Lens
- 4. Intelligence Cycle • No matter what methodology you use… Collect Distribute Store Analyze intelligence analysis is an iterative process.
- 5. Social Media: Intelligence Perspective • Social Media Intelligence is a combination of the best and worst features of: – HUMINT – OSINT – SIGINT HUMINT OSINT SIGINT
- 6. Social Media Analysis Goals • Provide value to the organization – turn data into intelligence using an “operational lens” • Ensure cyclical feedback occurs during collection, processing, analysis, and consumption • Validate that a particular network is the right source of data for the questions you need answered
- 7. Common Misconceptions • Social media is not a panacea – Not everyone uses social media – Users of social media use it unevenly – User behavior changes based on situations • Just because people can talk about anything does not mean they talk about everything all the time.
- 8. Common Pitfalls • Analyzing What Instead of Why: The important thing is often not what people are saying… but why they are saying it. • Using the Wrong Analysis Tools: Reporting tools rarely help dig into the why. Many common tools, reports, and metrics are actually misleading: – Word clouds atomize message context – Sentiment metrics are often highly inaccurate – Information in aggregate hides more than it reveals
- 9. Pitfalls: An Example of the Challenge
- 10. Pitfalls: An Example of the Challenge
- 11. Dangers of Disintegration Source: Matthew Auer, Policy Studies Journal, Volume 39, Issue 4, pages 709–736, Nov 2011
- 12. Analytic Framework • Data Capture (DC) Capture • Data Reporting (DR) • Data Analysis (DA) – What to measure Analyze Report – What the data is saying – What should be done based on the data Source: Avinash Kaushik, Occam’s Razor Blog http://www.kaushik.net/avinash/web-analytics-consulting- framework-smarter-decisions/
- 13. Choosing a Platform • Social media, and the ways that it is used, is relatively new and evolving rapidly: – Static approaches to social media are flawed from the outset – No one metric or set of metrics will always let you know what is happening • Platforms need to be open and highly adaptable to facilitate data capture, reporting, and analysis
- 14. Case Study: Brand Management • Industry: Gaming – Experiencing 10% growth annually – Overall revenue expected to exceed $80 billion by 2014 • In May, Zenimax Online Studios announced Elder Scrolls Online – Elder Scrolls V: Skyrim 2nd largest game of 2011
- 15. Problem Definition • Question: How can brand managers use social media to track and understand public attitudes toward a product? • Challenge: Capture relevant information for social media sources. – Query too large = false positives – Query too small = miss potential information
- 16. Twitter • Twitter has excellent analytical potential: – Enormous volume, 400 million+ tweets per day – Large user base, 140 million+ accounts – Open API • But its not without its limitations: – 140 characters – Limited historical (lookback) capacity without using a 3rd party provider like DataSift or GNIP
- 17. Data Capture: Initial Query • Twitter search for “Elder Scrolls Online” – Simplest possible way to access information – RSS feed for 10 days (Jun 27 – July 6 2012)
- 18. Data Capture: Entities & Associations Hashtag TwitterHandle URL Unstructured Keywords Time / Date Stamp Who What When Where TwitterHandle Hashtags, Keywords, Time, Date Geo (if Available) URLs
- 19. Data Reporting
- 20. Data Reporting
- 21. Data Analysis • Analysis needs to be rooted in the operational need: “How can I use social media to track and understand public attitudes toward my product” • Emphasis on hypothesis generation, testing, and experimentation
- 22. Data Analysis: Hashtags • Top hashtags were almost all generic or abstract – Undermines tracking and understanding – Top hashtags tied to franchise, not to the game Hashtags #ElderScrolls #concept #games #nerd #online #geek #MMO #gamer #skyrim #ScreenShot
- 23. Data Analysis: Expanding the Query • Hash tags from an initial subset of Tweets fed back into the initial query Initial Query Expanded Query Results Results Twitter Stream
- 24. Data Analysis: Sentiment • Sentiment analysis on small snippets of text like Tweets is generally poor • Follow and convert linked URLs into derivative sources • Larger text sources offer potential value with sentiment analysis that tweets alone cannot offer
- 25. Data Analysis: Sentiment • Top negative and positive sentiment scores can provide a glimpse into aggregate attitudes • Provide starting points for additional analysis
- 26. Next Steps: Shape the Conversation • Create and promote hashtags that help shape the conversation and make it easier to collect and analyze the Twitter stream
- 27. Next Steps: Segment the Data • Segment, or cluster, your data by: – User name or handle – Hashtags – Keywords – Geographic region to explore patterns and trends at the micro level versus the entire dataset
- 28. Next Steps: Segment the Data
- 29. Next Steps: Graph Analysis
- 30. Lessons Learned • Don’t: – Try drinking from a fire hose, sometimes less really is more; – Use metrics you can’t tie to actions; – Use visualizations or reports that strip the data from its context.
- 31. Lessons Learned • Do: – Segment data rather than attempting to work in the aggregate; – Look for the why behind the message; – Always return to the source material; – Explore alternative explanations; – Always consider the ultimate goal.
- 32. Thank You! Craig Vitter www.ikanow.com cvitter@ikanow.com github.com/ikanow/Infinit.e
Editor's Notes
- #2: Introduction and Topic
- #3: Agenda:Social Media: An Intelligence perspectiveCommon Analytic PitfallsAn Analytic FrameworkCase Study: Brand ManagementWays Forward, Future AnalysisQuestions?
- #4: Intelligence is information (data) that has been transformed to meet an operational need.There are a lot of ways to move from raw data to usable intelligence.
- #5: No matter what methodology you use…intelligence analysis is an iterative processYou Collect the data, Store it, Analyze it, and Distribute the end results to your organization in some usable format.
- #6: HUMINT, Human Intelligence: intelligence gathering by means of interpersonal contact. Pros: Can reveal intentions Cons: Can be unreliableOSINT, Open Source Intelligence: intelligence collected from publicly available sources. Pros: Fast and accessible Cons: NoiseSIGINT, Signals Intelligence: intelligence-gathering by interception of signals. Pros: High volume Cons: Noise
- #8: My mom does not Tweet or have a FaceBook profile. It only seems like your friends post or tweet every 30 seconds.For example, people use different networks for different reasons so tracking individuals consistently can be difficult
- #9: Why is someone tweeting or posting? If some checks in from a store is it really because the store is so incredible that they need to share that information or is because they are trying to form an impression about their lifestyle (i.e. image shaping)?Why is much harder than What.
- #10: http://apps.washingtonpost.com/politics/transcripts/2012/presidential/live/737/Washington Post and Votertide collaboration to analyze how viewers reacted to Clinton’s speech at the DNC Convention a few months ago.They captured 496,222 tweets and generated what amounts to a very basic word cloud that really provides limited value from an analysis perspective.
- #11: What can you learn from this type of experiment with the wrong tools?A lot of people were tweeting when Clinton was speaking but not many were really tweeting about what Clinton was saying.People like funny tweets
- #12: Word clouds can tell you something about the language used but not the meaning behind the language. What you see in the cloud is what but not why.
- #13: So how do you avoid some of these pitfalls and get useful intelligence from social media? The answer to that question, or partial answer at least is the focus of the remainder of the presentation.The key ingredients for a framework include Data Capture, Data Reporting and Data Analysis components. All of which are important but the Data Analysis components are the most interesting. :>
- #14: There are a lot of platforms that you could use to do social analysis but a few key issues to consider before making a commitment:
- #15: This is where I repeat that I shamelessly stole most of this presentation from a coworker Andrew who is a huge gaming fan as well as being a super bright analyst.
- #17: Almost all NLP/text extraction/unstructured data analysis tools perform poorly on small blobs of text
- #26: Negative: Users weren’t impressed by the game’s teaser and graphics suggesting that the trailer hadn’t been well received.Positive: Other hashtags showed that fans still had positive sentiment towards the Elder Scrolls franchise in general.
- #30: Use Graph Analysis to explore the links between entities extracted from your data, for example:Identify Key InfluencersView links between tweets, websites, and blogs