SlideShare a Scribd company logo
BigData
Architectures
Daan Gerits
Dasos
Volume
IOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOO
OIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOI
OIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOII
IOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIII
OIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIOII
OIOIOIIIOIOOOOIOIOOIIOIIIIIOIIOIIOIOIOIOIOIOIIOIIOIOIOIIIOIOOOOIOII

We already have that:
- NAS/SAN
- High Performance Computing
Variety

IOII IOIIIOIIIOII

IOII
IOII
IOII

IOII

IOII

We already have that:
- Meta-modeling
- NAS/SAN
Velocity

OIII
IOII
IOII OO
OIII
We already have that:
- Complex Event Processing
But do you have all of that in 1
platform?
But How??
Architectures

(Thx Nathan Marz!)
Analytical Big Data

Analysis Oriented
Optimize
Non-intrusive
Delta
Apps
Dashboards
Distributed
Database

Data
Sources
Ingestion
Engine

Enrich

Data
Systems
Delta
Impala,
Hive, ...

Apps
Dashboards

Distributed
Database

Data
Sources
Flume,
Sqoop,
Scribe, ...

MR, Pig,
Crunch,
Mahout, ...

MR, Pig,
Crunch, ...

Data
Systems
Delta

Analytical Big Data
architecture for enriching mostly
structured data with the goal to
optimize business processes.
Delta
Apps
Dashboards
Distributed
Database

Data
Sources
Ingestion
Engine

Enrich

Overload!

Data
Systems
Delta
Be
write-heavy
or
read-heavy
NOT both!
Operational Big Data
Focussed on Day-today business
Innovate
(Non-)intrusive
(Thx Nathan Marz!)
Lambda
Realtime
View A
Realtime
Processing

Apps

Realtime
View B

Dashboard

Realtime
View C

Data
Sources

Batch
View A
Fact
Store

Just In Time
Combiner

Batch
View B
Batch
View C

Reports
Lambda
Cassandra*
Storm

Apps

Cassandra*

Dashboard

Cassandra*
Custom
Code*

Data
Sources
ElephantDB
HDFS

ElephantDB
ElephantDB

Reports
Lambda

Operational Big Data
architecture for storing and processing

multi-structured and
immutable data with the goal to
Innovate business
Technologies to use

Pick your
stack!
Advice
Pilots, PoC, PoT, … do them!
Be pragmatic, start skinny
In Belgium: Variety > Volume
Be prepared to pivot on technologies
Questions?
Thoughts?
Ideas?
Disagreements?
...

daan.gerits@dasos.be
www.dasos.be
@daangerits

All images are used merely for illustrational means. In no
way was it my purpose to violate any rights by using
BigData
Architectures
Backup
Slides
Variety

Velocity

Volume
Lambda
Multistructured

Unstructured

Restructured

More Related Content

PDF
Storm @ Fifth Elephant 2013
PDF
Real-Time Analytics with Kafka, Cassandra and Storm
PDF
Don't Cross The Streams - Data Streaming And Apache Flink
PDF
Building large-scale analytics platform with Storm, Kafka and Cassandra - NYC...
PDF
Real Time Data Streaming using Kafka & Storm
PDF
A real time architecture using Hadoop and Storm @ FOSDEM 2013
PDF
Big Data Architecture
PPTX
Real time Analytics with Apache Kafka and Apache Spark
Storm @ Fifth Elephant 2013
Real-Time Analytics with Kafka, Cassandra and Storm
Don't Cross The Streams - Data Streaming And Apache Flink
Building large-scale analytics platform with Storm, Kafka and Cassandra - NYC...
Real Time Data Streaming using Kafka & Storm
A real time architecture using Hadoop and Storm @ FOSDEM 2013
Big Data Architecture
Real time Analytics with Apache Kafka and Apache Spark

Viewers also liked (7)

PPTX
Getting more out of your big data
PDF
Apache storm vs. Spark Streaming
PPT
Big Data
PDF
Kafka and Storm - event processing in realtime
PPTX
Big Data & Hadoop Tutorial
PDF
Hadoop Summit Europe 2014: Apache Storm Architecture
PPTX
Big data ppt
Getting more out of your big data
Apache storm vs. Spark Streaming
Big Data
Kafka and Storm - event processing in realtime
Big Data & Hadoop Tutorial
Hadoop Summit Europe 2014: Apache Storm Architecture
Big data ppt
Ad

Similar to Big data architectures (20)

PDF
Big Data Analytics: Architectural Perspective
PDF
Architecting Virtualized Infrastructure for Big Data
PDF
Big Data Architectures
PDF
Towards A Reference Architecture for BIG DATA.pdf
PDF
Architecting Modern Data Platforms Jan Kunigk Ian Buss Paul Wilkinson
PDF
Big Data/Hadoop Infrastructure Considerations
PPTX
MapR-DB – The First In-Hadoop Document Database
PDF
Introduction to big data and apache spark
PDF
High-performance database technology for rock-solid IoT solutions
PDF
big data analytics introduction chapter 1
PDF
Big Data Ecosystem
PDF
Big Data Architectures
PDF
Transform from database professional to a Big Data architect
PDF
Binder1.pdf
PDF
Big Data Architecture For enterprise
PDF
Business of Big Data
PPTX
MongoDB & Hadoop - Understanding Your Big Data
PPTX
Big Data Infrastructure and Hadoop components.pptx
PDF
Introduction to Big Data
PDF
MAZZ -Bob Towards BIG DATA-RA-AlloyCloud-NIST_BD.pdf
Big Data Analytics: Architectural Perspective
Architecting Virtualized Infrastructure for Big Data
Big Data Architectures
Towards A Reference Architecture for BIG DATA.pdf
Architecting Modern Data Platforms Jan Kunigk Ian Buss Paul Wilkinson
Big Data/Hadoop Infrastructure Considerations
MapR-DB – The First In-Hadoop Document Database
Introduction to big data and apache spark
High-performance database technology for rock-solid IoT solutions
big data analytics introduction chapter 1
Big Data Ecosystem
Big Data Architectures
Transform from database professional to a Big Data architect
Binder1.pdf
Big Data Architecture For enterprise
Business of Big Data
MongoDB & Hadoop - Understanding Your Big Data
Big Data Infrastructure and Hadoop components.pptx
Introduction to Big Data
MAZZ -Bob Towards BIG DATA-RA-AlloyCloud-NIST_BD.pdf
Ad

More from Daan Gerits (6)

PPTX
Apache kafka
PDF
Big Data BluePrint
PPTX
BigBoards.io Strata Ignite
PDF
IoT and BigData
PDF
Start small bigger biggest
PDF
Big data, why care
Apache kafka
Big Data BluePrint
BigBoards.io Strata Ignite
IoT and BigData
Start small bigger biggest
Big data, why care

Recently uploaded (20)

PDF
Getting Started with Data Integration: FME Form 101
PDF
Network Security Unit 5.pdf for BCA BBA.
PPTX
1. Introduction to Computer Programming.pptx
PDF
A comparative analysis of optical character recognition models for extracting...
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Machine learning based COVID-19 study performance prediction
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PPT
Teaching material agriculture food technology
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Empathic Computing: Creating Shared Understanding
PDF
Approach and Philosophy of On baking technology
PDF
Accuracy of neural networks in brain wave diagnosis of schizophrenia
PPTX
SOPHOS-XG Firewall Administrator PPT.pptx
PPTX
MYSQL Presentation for SQL database connectivity
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Assigned Numbers - 2025 - Bluetooth® Document
PPTX
A Presentation on Artificial Intelligence
Getting Started with Data Integration: FME Form 101
Network Security Unit 5.pdf for BCA BBA.
1. Introduction to Computer Programming.pptx
A comparative analysis of optical character recognition models for extracting...
Programs and apps: productivity, graphics, security and other tools
Machine learning based COVID-19 study performance prediction
“AI and Expert System Decision Support & Business Intelligence Systems”
Teaching material agriculture food technology
The Rise and Fall of 3GPP – Time for a Sabbatical?
Empathic Computing: Creating Shared Understanding
Approach and Philosophy of On baking technology
Accuracy of neural networks in brain wave diagnosis of schizophrenia
SOPHOS-XG Firewall Administrator PPT.pptx
MYSQL Presentation for SQL database connectivity
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
MIND Revenue Release Quarter 2 2025 Press Release
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Assigned Numbers - 2025 - Bluetooth® Document
A Presentation on Artificial Intelligence

Big data architectures