Fried data summit big data for lob content
2 likes507 views
- Big data can be used to analyze line-of-business content from various sources like documents, emails, databases to gain insights. - Text analytics techniques like entity extraction and fact extraction can be applied to structure unstructured human language data and discover relationships within it. - A successful approach involves connecting to authoritative data sources, deploying a user-friendly interface focused on specific use cases, and starting with a targeted application to incorporate business drivers.
Fried data summit big data for lob content
- 1. Big Data for Line-of-Business Content Jeff Fried CTO, BA Insight Data Summit May 2017
- 3. Big Data Is…
- 4. Focused on Search and SharePoint since 2004 Longtime Search Nerd • CTO, BA Insight • Senior PM, Microsoft • VP, FAST • SVP, LingoMotors About Jeff Fried Passionate About • Search • SharePoint • Search-driven applications • Information Strategy Blog: BAinsight.com/blog Technet Column “A View from the Crawlspace” jeff.fried@bainsight.com
- 5. About BA Insight – Connectivity – Applications - – Classification - – Analytics
- 6. This session
- 7. Process data Human data Machine data
- 8. Source: Forrsights Strategy Spotlight: Business Intelligence And Big Data Unstructured 50TB Semi-structured 2 TB Structured 12 TB Only 12% used today Average data volume per company 9 TB 75 TB 0.6 TB 5 TB 4 TB 50 TB SMBs: LEs: Companies don’t use most of their data
- 10. Connectors to Many Enterprise Systems • Aderant • Amazon S3 • Alfresco • Box • Confluence • CuadraSTAR • Elite / 3E • EMC Documentum • EMC eRoom • Google Drive • HP Consolidated Archive • (EAS, aka Zantaz) • HPE Records Manager/HP TRIM • IBM Connections • IBM Content Manager • IBM DB2 • IBM FileNet P8 • IBM Lotus Notes • IBM WebSphere • iManage Work • Jive • LegalKEY • LexisNexis Interaction • Lotus Notes Databases • Microsoft Dynamics CRM • Microsoft Exchange • Microsoft Exchange Public Folders • Microsoft SQL Server • MySQL • NetDocuments • Neudesic The Firm Directory • Objective • OpenText LiveLink/RM • OpenText eDOCS DM • Oracle Database • Oracle WebCenter • Oracle WebCenter Content (UCM/Stellent) • PLC/Practical Law • ProLaw • Salesforce.com • SAP ERP • ServiceNow • SharePoint Online • SharePoint 2016 • SharePoint 2013 • SharePoint 2010 • SharePoint 2007 • Sitecore • Any SQL-based CRM system • Veeva Vault • Veritas Enterprise Vault (Symantec eVault) • West km • Xerox DocuShare • Yammer
- 12. The average $1 billion company maintains 48 disparate financial systems and uses 2.7 ERP systems Integration Gaps Impact Performance Source: The Hackett Group
- 13. Big Data on LoB Data: Data Science and Data Discovery • Volume, velocity, and variety of data • Potential business impact • Ease of use • Agility and flexibility • Time-to-results • Installed user base • Complexity of analysis • Potential impact • Range of tools • Smart algorithms • Difficult to implement • Slow and complex • Narrow focus of analysis • Limited depth of information exploration • Low complexity of analysis BIG DATA DATA SCIENCE DATA DISCOVERY
- 14. Source: PARC Predictive Inventory Levels to Minimize Warehousing Costs Personalized Medicine Treatment Programs Smart Meter Monitoring for Customer Value Add Customer Churn Analysis for Increased Customer Lifetime Value Trade Options and Futures Pricing Platform
- 15. – o o – – – Example: Optimizing Leaseholds & Mineral Rights in Oil & Gas Exploration
- 16. – o o – – – Example: Clinical Trial Management for Pharmaceutical Development
- 17. Example: Genetics Analysis for Clinical Research 18
- 18. – – – – Example: Insurance Claim Analysis I am a software designer and sit at a desk all day. I could not sit comfortably for months. I was unable to work until the beginning of November. However, my company could not wait for me to recover and was not able to provide me with my job back. I did have six months of short term disability, which I have to repay. I was unable to get a new job until January 2, so I will be claiming lost earnings from April 4 until the end of December, nine months worth.
- 20. Example: (Pharma R&D) Unified View 1. Documentum Image 2. SharePoint Doc 3. Regulatory Record 4. MEDLINE article Multiple Sources One View Search: amgen 655 Relationships Discovered: Antibodies: mAb Receptors: DR5, IGF-1R Labs: Oncology 1 People: David Chang
- 21. Examples: Financial Risk Management
- 23. Data Discovery Applications - Patterns Research Portal Unified View Customer Service Compliance Analyst’s workbench Management Adviser Innovation Center Voice of the Customer Logistics Center Consolidated Dashboard Call Center Online Service Sales Dashboard Fraud Center E-Discovery Info Governance
- 26. A “Recipe” for harnessing LoB data Connect to Authoritative Sources Develop a list, prioritize, and iterate Create Structure from Human Language Using text analytics techniques Deploy a polished, flexible UX Focus on users and use cases, don’t over-constrain it Start with a Target Application Incorporate your business drivers
- 27. Selecting Sources a b c d e f g h ij k l mn o p q ImpactofContent Onboarding & Cleanup Effort Content Sources for Onboarding a R&D projects - reports b R&D projects - research notebooks c Historical projects (OCR) d Prototype data e Lab notes f Patent prep library g CAD drawings h Testing/Stress Data i Design Patterns j Technical Data Sheets k Expert Profiles l Regulation Database m Subscription (OneSource, Lexis) n Industry database o Competitor Web Crawling p Industry patents q Newswires
- 29. Entity Extraction • Well Established • Often essential to faceted navigation • Driven by lexical resources (Taxonomies) Acronym Person Location End of sentence End of paragraph Date Base = 2002-03-XX
- 30. Fact Extraction Substance Base=„Gold“ Class=„Element“ Number=79 Symbol=Au Location Base=„Qilian“ Country=„China“ Region=„Asia“ Subregion=„East“ „The Red Valley property lies within the Qilian fold belt which is host to gold deposits.“ Qilian is location of gold Extracted Fact: Substances x Locations Substance Base=„Gold“ Class=„Element“ Number=79 Symbol=Au Location=„Qilian“ Location Base=„Qilian“ Country=„China“ Region=„Asia“ Subregion=„East“ Substance=„Gold“ Indicates a gold location
- 31. 32 User Experience is a Discipline
- 32. 33
- 33. This session
- 35. Traditional IM Requirements based Top-down design Integration and re-use Technology Consolidation World of EDW, CRM, ERP, ECM Competence Centers Commercial Software “Big Data” Style Opportunity Oriented Bottom-up Experimentation Immediate use and gratification Tool proliferation “World of Hadoop” Hackathons Open Source