Not about the Big in Big Data
2 likes423 views
This document discusses how big data is not just about data volume, but also about data variety, velocity, and complexity. It argues that traditional data warehousing and business intelligence approaches are no longer sufficient, and that new approaches using Hadoop and NoSQL databases are needed to enable real-time insights from both structured and unstructured data at scale. These new approaches can help organizations gain a 360-degree view of their business and make faster, more data-driven decisions.
Not about the Big in Big Data
- 1. It’s Not About the “Big” in Big Data
- 2. $26.5 BILLION Nick Leeson Jerome Kerviel Kweku Adoboli Bruno Iksil Barings Bank Societe Generale UBS JPMC $1.4 billion $6 billion $1.4 billion $17.5 Billion Nasa $17.6B Rogue Traders $26.5B
- 4. Hadoop 2012
- 5. Hype Cycle We are here
- 6. Crossing the Chasm We are here
- 8. Database Harddrive Unstructured ( 61.7% growths ) time to find one record = logb N * 10ms log100(100,000,000) * 10ms = 40ms time to read record = 10ms 10,000,000 * 50ms = 5.8 days
- 9. Hadoop Harddrive throughput = 10MB/s time to transfer record = 10ms 10,000,000 * 10ms = 1.5 days random reads = (5.8 days)
- 10. Laws of Physics Random Sequential Values/Sec. 316 Disk 53,200,000 1,924 SSD 42,200,000 36,700,000 Memory 358,200,000 1 10 100 1,000 10,000 100,000 1,000,000 10,000,000 100,000,000 1,000,000,000 Adam Jacobs The Pathologies of Big Data
- 12. Time to insight Time for decision
- 13. Volume Data Size Data Complexity Sp ang es So ta Ch ee e urc Da do ety Ve f ri loc Va y it
- 15. Game Changer Slow Static Barrier Business ETL Data Warehouse Intelligence Fast Dynamic View Raw Load Hadoop Data Pipeline
- 16. NO - SQL RDBMS Standard SQL Structured Data Response in sec. No SQL Unstructured Data Batch Hadoop
- 17. Common Applications Asset Management Analytics Security Analytics Product Cohort Analytics Advanced Web Analytics Structured + Many Unstructured Decision Makers Data Sources Data
- 18. follow us: @datameer @stefanGroschupf