Operating in a Multi-execution Engine Hadoop Environment by Erik Halseth of Datameer
0 likes731 views
Datameer aims to simplify big data analytics with features such as wizard-led data integration, a point-and-click analytics interface, and drag-and-drop visualizations accessible on any device. The platform offers optimized mapreduce and in-memory processing to enhance efficiency, addressing limitations of traditional data analytics methods. Their solution integrates various data sources and provides sophisticated analytics without requiring complex ETL processes.
Operating in a Multi-execution Engine Hadoop Environment by Erik Halseth of Datameer
- 2. © 2014 Datameer, Inc. All rights reserved. Datameer’s Vision! Make big data analytics simple for everyone
- 3. © 2014 Datameer, Inc. All rights reserved. What Datameer Offers! Wizard-led Data Integration! • No ETL! • 59 Connectors + plug-in API! • Smart Sampling! Point-and-click Analytics! • Interactive spreadsheet UI! • 270 pre-built analytic functions! • Macros & function plug-in API! Drag-and-Drop Visualization! • Blank canvas for design ! • HTML5, consumable on any device! • Visualization plug-in API!
- 4. © 2014 Datameer, Inc. All rights reserved. Smart Analytics! Column Dependencies Decision Tree Recommendation Engine Clustering
- 5. © 2014 Datameer, Inc. All rights reserved. Where does Datameer sit?!
- 6. © 2014 Datameer, Inc. All rights reserved. Classic Business Analytics Data Flow !
- 7. © 2014 Datameer, Inc. All rights reserved. New Business Analytics Data Flow!
- 8. © 2014 Datameer, Inc. All rights reserved. Datameer On Premise Installation!
- 9. © 2014 Datameer, Inc. All rights reserved. Datameer Implementation - Cloud!
- 10. © 2014 Datameer, Inc. All rights reserved. ! Stefan Groschupf CEO, Co-Founder! Problem
- 11. © 2014 Datameer, Inc. All rights reserved. Typical Data Analytics Funnel Raw Data (TB-PB) Insights (KB) ! More sophisticated ! Less change ! High value ! Power users ! Planned / scheduled ! More ad hoc ! More change ! High & low value ! Casual users ! Interactive sessions 5 - 15 steps, iterative algorithms Explore Summarize Prepare Learn Aggregate Present Slice
- 12. © 2014 Datameer, Inc. All rights reserved. Raw Data (TB-PB) Insights (KB) Map Reduce • Inefficient for small data! • High latency! Current Approaches: Either - Or Raw Data (<TB) Insights (KB) In-Memory • Only small data! • Very expensive! • Not Hadoop! Not New
- 13. © 2014 Datameer, Inc. All rights reserved. Small Data, Big Machine VS
- 14. © 2014 Datameer, Inc. All rights reserved. 600h Spent on Jobs < 100MB!
- 15. © 2014 Datameer, Inc. All rights reserved. ! Stefan Groschupf CEO, Co-Founder! Our Solution
- 16. © 2014 Datameer, Inc. All rights reserved. Smart Execution Raw Data (TB-PB) Insights (KB) New Optimized MapReduce In-Memory Single Node
- 17. © 2014 Datameer, Inc. All rights reserved. Architecture Hadoop MapReduce Dataflow Graph Engine YARN Smart Execution Engine In-Memory Tez Others Data Integration Visualization Spreadsheet Other (SQL) Single Node
- 18. © 2014 Datameer, Inc. All rights reserved. Workflow Data Sets System Resources Optimized! MapReduce! Single Node! In-Memory! Future! Technology! Analytics
- 19. © 2014 Datameer, Inc. All rights reserved. DAG Processing vs.!
- 20. © 2014 Datameer, Inc. All rights reserved. Transparent for End Users
- 21. @Datameer!