Getting more out of your big data
Download as PPTX, PDF0 likes681 views
The document discusses how cloud computing is creating new business opportunities through processing large amounts of data quickly. Some key points discussed include: - Processing 3.5 billion events per day to increase ad revenue and 3 billion signals per day to measure online influence. - Processing 464 billion rows per quarter with average query times under 10 seconds to handle massive volumes of data. - Using tools like sentiment analysis and web analytics on internal cloud systems to gain operational efficiencies. - Improving operational decision making for IT managers and users through real-time insights generated from big data analysis.
Getting more out of your big data
- 2. Cloud
- 3. Creating New Business Opportunities Revenue Growth Increases ad revenue by processing 3.5 billion events per day Massive Volumes Processes 464 billion rows per quarter, with average query time under 10 secs. Business Innovation1 Measures and ranks online user influence by processing 3 billion signals per day Cloud Connectivity Connects across 13 social networks via the cloud for data and API access Operational Efficiencies Uses sentiment analysis and web analytics for its internal cloud GE Real-Time Insight Improves operational decision making for IT managers and users 1. Klout Case Study: http://www.microsoft.com/casestudies/Microsoft-SQL-Server-2012-Enterprise/Klout/Data-Services-Firm-Uses-Microsoft-BI-and-Hadoop-to-Boost-Insight-into-Big-Data/710000000129
- 7. Big Data Creating Transparency Enabling Experimentation to discover needs, expose variability and improve performance Segmenting populations to customize actions Replacing/Supporting human decision making with automated algorithms Innovating new business models, products and services with big data
- 9. HADOOP Data Repository Internal Data through APIs Fetcher & Parser to enrich & validate with external data Data Silos High Velocity changes Cost of Changes BusinessChallenges Solution SolutionBenefits Faster Updates Full dataset refresh possible every week instead of a few times per year Cost Reduction Significant reduction in validation phone calls SolutionBenefits
- 10. HADOOP Real-Time Architecture Scaleable architecture to support current and future real-time insight needs High Volume & High Velocity Old solution not able to handle incoming volumes of data in timely manner BusinessChallenges Solution SolutionBenefits Faster Insights Realtime handling of the growing Volume & Velocity of the data. Adding at least 1TB per year. Grow with Needs Solution scales with business needs without upfront cost SolutionBenefits
- 11. HADOOP Data & Processing Cluster Scalable Image Library Processing cluster to process images High Variety & High Volume Analyses of 30.000+ giant images of medical scans of Pancreas BusinessChallenges Solution SolutionBenefits Faster Research & Diagnostical Insight Diagnoses can be validated against previous diagnoses New research ideas can be checked across full image set Cost Friendly Reliability Improvement Inexpensive data duplication over HADOOP storage nodes provides needed reliability improvements SolutionBenefits
- 12. Share your data with the world via Azure Marketplace Enrich with social media data via Social Analytics Advanced analytics with Hadoop Connecting with the World’s Data Analyze Big Data with familiar tools Immersive insights from any data JavaScript based simple programming Immersive Insight, Wherever you are Simplicity and manageability of Windows to Hadoop Extended data warehousing with Hadoop Scale & elasticity of cloud Any Data, Any Size Anywhere HDInsight - Microsoft’s approach to Big Data
- 13. Hive Excel Plugin, ODBC Driver integrates Hadoop to SQL Server Analysis Services, PowerPivot, and Power View Familiar BI tools with structured and unstructured data BenefitsKeyFeatures
- 14. Integration with enterprise BI solutions Microsoft SQL Server connector for Apache Hadoop with SQOOP (SQL to Hadoop) Integration with Microsoft Enterprise Data Warehouses SQL Server Parallel Data Warehouse connector for Apache Hadoop with SQOOP Deeper insights from structured and unstructured data BenefitsKeyFeatures
- 15. Unlock rare patterns from bespoke data mining models Support for open source predictive analytics tools such as R and Mahout New business insights with predictive analytics from Microsoft Hive ODBC Driver connects Hadoop to SQL Server Data Mining tools BenefitsKeyFeatures
- 16. Mashing up of internal and public data sets via Data Explorer Integration with third-party data and services Sharing of data and insights through Windows Azure Marketplace Integration with Windows Azure Marketplace BenefitsKeyFeatures
- 17. Integration of social information with business applications Social Analytics Stronger customer relationships Integration with social media sites Models augmented with publicly available data from social media sites BenefitsKeyFeatures
- 18. MicrosoftHDInsight Enterprise-class security Integration with Microsoft System Center Integration with Windows Server® Active Directory Simplified management of Hadoop on Windows Smart packaging of Hadoop on premises Fast deployment of Hadoop on Azure 100% Microsoft Support Easy setup on-premises and in the cloud BenefitsKeyFeatures
- 19. MicrosoftHDInsight Elastic peta-scale analytics on Microsoft’s cloud platform Hadoop-based Service on Windows Azure platform Enterprise-class Big Data platform on-premises Hadoop-based distribution on Windows Server BenefitsKeyFeatures
- 25. 1. Take a large problem and divide it into sub-problems 2. Perform the same function on all sub-problems 3. Combine the output from all sub-problems … … Output MAPREDUCE DoWork() DoWork() DoWork() …
- 32. spanning relational and non-relational Worlds NON-RELATIONAL 1 0 0 11 1 DATA MANAGEMENT SHARE AND GOVERN DISCOVER AND RECOMMEND TRANSFORM AND CLEAN INSIGHTS DATA ENRICHMENT OPERATIONAL SELF-SERVICE MOBILE PREDICTIVE REAL-TIME COLLABORATIVE MARKETPLACE ExternalData andServices RELATIONAL MULTIDIMENSIONAL STREAMING
- 33. Objectives Starter Offer Structured (+- 4/5weeks) engagement that demonstrates the capabilities of the Microsoft Big Data platform with a prototype using real customer data Who Delivers Microsoft Consulting Services & Industry Experts Expected Outcome Define Big Data Company Strategy Implement Big Data Prototype solution Customer Meeting to discuss Big Data Needs & Scoping for Starter Offer Scoping
- 34. At your service