Text, Content, and Social Analytics: BI for the New World
Download as PPTX, PDF2 likes1,585 views
This document discusses how text, content, and social analytics can enable business intelligence for the new world. It argues that analytics can extract insights from both structured and unstructured data sources, including text, content, and social media. Semantics plays a key role by capturing meaning, relationships, context and understanding from unstructured information. The integration of these different data types through analytics and semantics allows organizations to gain new perspectives from questions about public opinions, trends, and relationships. The presentation envisions continued progress towards more intelligent computing through combining big data, analytics, semantics and other techniques.
![“In this example, you can quickly see that the Drooling Dog Bar B Q has gotten lots of positive reviews, and if you want to see what other people have said about the restaurant, clicking this result is a good choice.”-- http://googleblog.blogspot.com/2009/05/more-search-options-and-other-updates.html“In the recap of [Searchology] from Google’s Matt Cutts, he tells us that: ‘If you sort by reviews, Google will perform sentiment analysis and highlight interesting comments.’-- Bill Slawski, “Google's New Review Search Option and Sentiment Analysis,” http://www.seobythesea.com/?p=1488](https://image.slidesharecdn.com/tdwidc-2011july15-110715055919-phpapp01/85/Text-Content-and-Social-Analytics-BI-for-the-New-World-12-320.jpg)
![My 2009 text-analytics market survey asked, [What information] do you need (or expect to need) to extract or analyze:Text Analytics 2009: User Perspectives on Solutions and Providers](https://image.slidesharecdn.com/tdwidc-2011july15-110715055919-phpapp01/85/Text-Content-and-Social-Analytics-BI-for-the-New-World-19-320.jpg)
Text, Content, and Social Analytics: BI for the New World
- 1. Text, Content, and Social Analytics: BI for the New WorldSeth GrimesAlta Plana Corporation@sethgrimesTDWI – Washington DCJuly 15, 2011
- 2. Table of Content:Principles.Perspectives.Semantics.Text/content analytics.Social.BI for the New World.
- 3. Imperatives for the 2010s:Do more with more.“It’s Not Information Overload. It’s Filter Failure”: Clay Shirky, 2008.More sources & types of data.
- 5. New hardware and methods.Automate more, more intelligently.Analytics.
- 7. I see three categories of data:Quantities, whether measured, observed, or computed.Content, which I’ll characterize as non-quantitative information.Metadata (semantic & structural) describing quantities and content.Our concern is content, analytics & fusion.
- 8. Structured/unstructured is a false dichotomy.
- 9. Where do relationships fit?DW & BI relate numbers......but by-the-numbers BI lacks doesn’t explain.
- 10. Questions for business (& government):What are people saying? What’s hot/trending?What are they saying about {topic|person|product} X?... about X versus {topic|person|product} Y?How has opinion about X and Y evolved?How has opinion correlated with {our|competitors’|general} {news|marketing|sales|events}?What’s behind opinion, the root causes?Who are opinion leaders?How does sentiment propagate across multiple channels?
- 11. The answers are here...But how do you get at them?
- 12. “In this example, you can quickly see that the Drooling Dog Bar B Q has gotten lots of positive reviews, and if you want to see what other people have said about the restaurant, clicking this result is a good choice.”-- http://googleblog.blogspot.com/2009/05/more-search-options-and-other-updates.html“In the recap of [Searchology] from Google’s Matt Cutts, he tells us that: ‘If you sort by reviews, Google will perform sentiment analysis and highlight interesting comments.’-- Bill Slawski, “Google's New Review Search Option and Sentiment Analysis,” http://www.seobythesea.com/?p=1488
- 14. Analytics is a collection of tools and techniques that extract insights from data.Apply or embed analytics within business contexts – collect data and information about customers, markets, suppliers, and business processes – use results to inform, drive, and optimize business decision making – and you harness analytics as a core BI asset.
- 15. Analytics seeks structure in “unstructured” sources.x(t) = t y(t) = ½ a (et/a + e-t/a) = acosh(t/a)http://www.tropicalisland.de/NYC_New_York_Brooklyn_Bridge_from_World_Trade_Center_b.jpghttp://en.wikipedia.org/wiki/Seven_Bridges_of_K%C3%B6nigsberg
- 16. Text analytics models text.“Statistical information derived from word frequency and distribution is used by the machine to compute a relative measure of significance, first for individual words and then for sentences.”-- H.P. Luhn, The Automatic Creation of Literature Abstracts, IBM Journal, 1958.http://wordle.net
- 17. Document input and processingKnowledge handling is keyDesk Set (1957): Computer engineer Richard Sumner (Spencer Tracy) and television network librarian Bunny Watson (Katherine Hepburn) and the "electronic brain" EMERAC.Hans Peter Luhn“A Business Intelligence System”IBM Journal, October 1958
- 18. “This rather unsophisticated argument on ‘significance’ avoids such linguistic implications as grammar and syntax... No attention is paid to the logical and semantic relationships the author has established.” -- H.P. Luhn
- 19. My 2009 text-analytics market survey asked, [What information] do you need (or expect to need) to extract or analyze:Text Analytics 2009: User Perspectives on Solutions and Providers
- 21. From document to DB; an IBM example: “The standard features are stored in the STANDARD_KW table, keywords with their occurrences in the KEYWORD_KW_OCC table, and the text list features in the TEXTLIST_TEXT table. Every feature table contains the DOC_ID as a reference to the DOCUMENT table.”
- 22. Welcome to the New World.The Far Side by Gary LarsonKen Jennings, IBM Watson, and Brad Rutter play Jeopardy!https://secure.wikimedia.org/wikipedia/en/wiki/File:Watson_Jeopardy.jpg
- 23. In a sense, text analytics, by generating semantics, bridges search and BI to turn Information Retrieval into Information Access.Information AccessSearchBIText AnalyticsIntegrated analyticsSemantic search
- 24. Have we arrived?2001: A Space Odyssey, Stanley Kubrick
- 26. Intelligent computing involves:Big (and little) Data.Quantities.
- 27. Content.
- 29. Semantics enables better content production, management & use. Semantics captures –MeaningRelationshipsContext Understanding–the sense of “unstructured” online, social, and enterprise information, for content consumers and publishers.Semantics unites data of all types.
- 33. From connections to influence: What’s wrong with these pictures? (Radian6, Sysomos, Klout)
- 34. Social analytics:Use social data in analyses (alongside enterprise & online information).Content.
- 35. Connections.Bring BI to social analyses.3rd & 4th senses of social analytics:Adopt agile, collaborative methods.Share your data.A challenge: Enterprise-social-online data integration.
- 36. Text, Content, and Social Analytics: BI for the New WorldSeth GrimesAlta Plana Corporation@sethgrimesTDWI – Washington DCJuly 15, 2011