Azure Big Data Story
Download as PPTX, PDF3 likes1,884 views
This document discusses Azure big data capabilities including the 5 V's of big data: volume, velocity, variety, veracity, and value. It notes that 60% of big data projects fail to move beyond pilot according to Gartner. It then provides details on Azure persistence choices for storing big data including storage, Data Lake, HDInsight, DocumentDB, SQL databases, and Hadoop options. It also discusses load and data cleaning choices on Azure like Stream Analytics, SQL Server, and Azure Machine Learning. Finally, it presents 5 architectural patterns for using Azure big data capabilities.
Azure Big Data Story
- 1. Azure Big Data Story @LynnLangit
- 2. I love to…Learn & Build
- 4. Azure Big Data by the V’s Value Volume Velocity Variety Veracity
- 5. Big Data = Business Value ? 60% of Big Data projects FAIL to go beyond pilot and will be abandoned (through 2017) - Gartner
- 6. Volume How big is Big Data?
- 7. Variety Is my Data rectangular?
- 10. Azure Persistence Choices • Storage • Store Simple Files • Data Lake • HDInsight • Cloudera Hadoop • MANY… NoSQL • SQL Azure • SQL Azure DW • SQL Server on Azure VM Relational
- 11. Drilling In: Relational AND NoSQL
- 12. Azure Persistence Choices Detailed • Storage • Store Simple Files • Data Lake • HDInsight • Cloudera Hadoop • Redis Caching • DataStax Enterprise • Document DB • Mongo Labs • Graph Engine NoSQL • SQL Azure • SQL Azure DW • SQL Server on Azure VM Relational
- 13. Velocity How fast is my Data?
- 14. Veracity How clean is my Data?
- 16. Azure Load Choices Load Libraries StreamEvent Hub Batch Stream Analytics
- 18. Azure Data Cleaning Choices ETL • SQL Server VM • Data Pipeline Client • Power BI • Power Query ML • Azure ML • Data Marketplace • SQL Server DQS
- 19. Data Pipelines
- 23. Data Model On Premise SQL Server+ Cloud Azure+ Key-Value Queues None Windows Queues Azure Redis Cache Azure Queues Wide Sparse Columns Columnstore Index SSAS Tabular Models Azure Tables DataStax Enterprise (Cassandra) Files FileTable, Filestream XML data type Azure BLOB Store StoreSimple JSON or Graph SQL Server 2016 None Azure DocumentDB / Graph Engine (beta) Hosted MongoDB or Neo4J Large Relational SQL Server Enterprise PDW SQL Analysis Services SQL Database (basic, standard, premium) APS SQL Data Warehouse Hadoop Hortonworks HDInsight/ Data Lake, Hosted Cloudera Other Stream Insight Event Hub, StreamAnalytics, ML Marketplace
- 25. Value How useful is my Big Data?
- 26. Big Data = Business Value ? 60% of Big Data projects FAIL to go beyond pilot and will be abandoned (through 2017) - Gartner
- 29. Architecture 1- File Storage / Backup
- 30. Architecture 2- Data Warehouse
- 31. Architecture 3 – Operational Database
- 32. Old becomes New
- 33. Architecture 4 – Small Big Data
- 34. MORE DATA
- 35. Architecture 5 – Big Data
- 37. Azure Big Data Story @LynnLangit
Editor's Notes
- #3: https://about.me/LynnLangit
- #4: ScottGu Blog - http://weblogs.asp.net/scottgu Azure Big Data -- http://azure.microsoft.com/blog/topics/big-data/ Data Factory Pipeline Sample (Blog) -- http://azure.microsoft.com/blog/2015/04/24/azure-data-factory-update-simplified-sample-deployment/
- #6: http://www.bain.com/publications/articles/three-promises-and-perils-of-big-data.aspx
- #12: http://azure.microsoft.com/en-us/documentation/infographics/building-real-world-cloud-apps/
- #20: Image credit - http://tapoueh.org/images/pipeline.png Azure Data Factory tutorial - http://azure.microsoft.com/en-us/documentation/articles/data-factory-get-started-using-editor/
- #21: Image credit - http://tapoueh.org/images/pipeline.png Azure Data Factory tutorial - http://azure.microsoft.com/en-us/documentation/articles/data-factory-get-started-using-editor/
- #22: ScottGu Blog - http://weblogs.asp.net/scottgu Azure Big Data -- http://azure.microsoft.com/blog/topics/big-data/ Data Factory Pipeline Sample (Blog) -- http://azure.microsoft.com/blog/2015/04/24/azure-data-factory-update-simplified-sample-deployment/
- #25: http://recode.net/2014/04/15/microsofts-big-data-angle-office-as-a-friendly-front-end/
- #27: http://www.bain.com/publications/articles/three-promises-and-perils-of-big-data.aspx
- #28: http://www.wsj.com/articles/ubers-new-funding-values-it-at-over-41-billion-1417715938
- #33: http://www.nytimes.com/newsgraphics/2013/08/18/reshaping-new-york/
- #35: www.thingful.net