Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and Analytics
- 1. Tuesday, May 1, 12
- 2. Eric.kavanagh@bloorgroup.com Twitter Tag: #briefr Tuesday, May 1, 12
- 3. Reveal the essential characteristics of enterprise software, good and bad Provide a forum for detailed analysis of today’s innovative technologies Give vendors a chance to explain their product to savvy analysts Allow audience members to pose serious questions... and get answers! Twitter Tag: #briefr Tuesday, May 1, 12
- 4. May: Analytics June: Intelligence July: Governance August: Analytics Twitter Tag: #briefr Tuesday, May 1, 12
- 5. Ultimately analytics is about businesses making optimal decisions, although the range of technologies that inhabit this area is wide: statistical analysis, data mining, process mining, predictive analytics, predictive modeling, business process modeling and additionally complex event processing. With the advent of big data, analytics has become “big analytics” with organizations diving into large heaps of data that previously was not available or usable. Open source technologies (Hadoop, etc.) in conjunction with the cloud have expanded the range of what is possible in the cloud and considerably reduced the price of leveraging new and, often very substantial data sources. Twitter Tag: #briefr Tuesday, May 1, 12
- 6. Robin Bloor is Chief Analyst at The Bloor Group. Robin.Bloor@Bloorgroup.com Twitter Tag: #briefr Tuesday, May 1, 12
- 7. Pervasive Software, a provider of data integration and database software, introduced Pervasive DataRush, a parallel data flow development platform several years ago. Aside from marketing that capability it has been using it to build data integration and data flow enabled BI products that exploits the DataRush capability. Pervasive RushAnalyzer is one the new parallel BI products that has been built using DataRush. It is aimed squarely at solving problems of in the management and analysis of big data, and delivering new capabilities. Twitter Tag: #briefr Tuesday, May 1, 12
- 8. David Inbar is Senior Director, Pervasive Big Data Products & Solutions leading the business and product management functions for Pervasive’s Big Data Products group. Previously he led the global marketing and international channels teams for Pervasive’s Integration Products group as well as the company’s Innovation Lab. David has driven innovative business models and technology adoption strategies for many application development and data management products. Jim Falgout is Chief Technologist, Pervasive Big Data Products and Solutions. As Chief Technologist for Pervasive’s Big Data team, Jim Falgout is responsible for setting innovative design principles that guide Pervasive engineering teams as they develop new big data-focused releases and products. Jim is responsible for the architectural design of a software development platform for parallel applications that deliver high throughput on big data. Twitter Tag: #briefr Tuesday, May 1, 12
- 9. May 1, 2012 Drinking from the Fire Hose: Practical Approaches to Big Data Preparation and Analytics The Briefing Room bigdata.pervasive.com
- 10. The Internet is the Fuel for the Fire Source: IBM Corporation 2
- 11. The Real Culprit: an Internet of Things Source: McKinsey Global Institute report on Big Data, May 2011 3
- 13. Big Data Pain Points :"##&(*, -.&/0.&, 730#+8&, :"34$%&, %"3)*"., /."1#&, 40%/#&,, .&/".*, #"5, ,%0*(2, %"6&#, (20.*, )35&4*, ,(#&034&, ,,6)4("9&., 6042;"0.6, &9&3*,(0/*$.&, ,,055.&50*&, 9)4$0#)8&, ,,0#&.*, 6&(.+/*, 0$6)*, /.&6)(*, (#"4&6,#""/, !"#$%&'!&#"()*+, <0*0,C3*&5.0*".4, ?$4)3&44,730#+4*4, <0*0,=()&3>4*4, <&()4)"3,@0A&.4, 7//,<&9&#"/&.4, <0*0,730#+4*4, B/&.0>"30#,C3*&##)5&3(&, 5
- 14. Time to Insight Falling Behind Data Growth 6
- 15. Big Data Analytics Software Requirements Additional Requirements •  Must be usable by business users and analysts •  Graphical/visual environment •  Option to extend via scripting •  Scalable and cross-platform: laptop, desktop, Hadoop cluster 7
- 16. 8
- 17. DEMO 9
- 18. Pervasive RushAnalyzer: Big Data Prep & Analytics 10
- 19. Pervasive RushAnalyzer Key Differentiators !  Comprehensive ETL and data preparation !  Analytics data scientists will love: machine learning !  Works with existing toolsets !  No cost to get started !  Scales from laptop to server to Hadoop clusters !  True distributed computing on Hadoop clusters 11
- 20. Twitter Tag: #briefr Tuesday, May 1, 12
- 21. Tuesday, May 1, 12
- 22. At the moment Big Data is often managed as “a project on the side” - isolated from the normal data flows associated with data warehousing This situation will not last. Either the large data heaps are ephemeral or they are here to stay. But once your start gathering data you don’t usually stop treated. If the big data heaps are here to stay they require data flow architecture. In that sense the Hadoop - Hive- HBase- Pig arrangement is really just a big prototype. That data flow architecture must serve both big data analysis and traditional data warehousing. Tuesday, May 1, 12
- 23. Tuesday, May 1, 12
- 24. We not only have the challenges of big data and big data flow, we also have the problem of data pool proliferation and the opportunities provided by data mashup/discovery If we extrapolate from now we run into a complexity of data flows that can no longer be managed by point-to- point thinking. In effect we get a combinatorial explosion - which dictates the need - in fact the necessity - for data flow architecture and data analysis architecture. If it didn’t deliver value, no-one would do it. Tuesday, May 1, 12
- 25. The PC Revolution, The Internet Revolution, The mobile revolution were all surprises even for those who saw them coming. They all brought more data and more data distribution. The coming Embedded revolution could be characterized as “the web of intelligent things” - things that know their state, report their state, can respond to their state or can respond collectively. Think of: A cup that knows what’s in it A house that knows whose home A car that knows how much you had to drink Tuesday, May 1, 12
- 26. The Challenge is Speed and Complexity Big Data has only just begun: Think of current big data projects as the early spreadsheets Data flow architecture is already an issue. Complexity is increasing Speed is the enabler or the barrier Twitter Tag: #briefr Tuesday, May 1, 12
- 27. Questions It is not clear to me what product classification this falls under. It appears to be a data flow architecture design and implementation capability. Is that the case? What does RushAnalyzer complement? What does it compete with? What interfaces does it have to different data sources? Clearly this is very fast operationally, because of the underlying parallelism. Can you give us some idea of how this compares in speed terms with, for example, a Hadoop arrangement aimed at a similar set of capabilities What skills are required to make best use of this capability? Twitter Tag: #briefr Tuesday, May 1, 12
- 28. Questions Who have been the early adopters of this kind of capability and what kind of business problems are they trying to solve? Which vertical business sectors have shown most interest and which have shown least interest? Quo vadis? Twitter Tag: #briefr Tuesday, May 1, 12
- 29. Tuesday, May 1, 12
- 30. May: Analytics • June: Intelligence • July: Governance • August: Analytics Twitter Tag: #briefr Tuesday, May 1, 12
- 31. Tuesday, May 1, 12