SlideShare uma empresa Scribd logo
Thiago de Faria
DataOps & AI Lead @LINKIT
@thiagoavadore#ITNEXT
NATIVE
STORAGE,
EPHEMERAL
CLUSTERS AND
THE PITFALLS
OF BUILDING
YOUR DATA
PIPELINE IN A
CLOUD WORLD
#ITNEXT @thiagoavadore
CLOUD.

BIG DATA.
MACHINE LEARNING.

AI.
#ITNEXT @thiagoavadore
CLOUD.

BIG DATA.
MACHINE LEARNING.

AI.
@thiagoavadore#ITNEXT
@thiagoavadore#ITNEXT
@thiagoavadore#ITNEXT
@thiagoavadore#ITNEXT
#ITNEXT @thiagoavadore
NOT A FRAMEWORK.
NOT A LANGUAGE.
NOT AN OS.
A BEAST ON ITS OWN.
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
@thiagoavadore#ITNEXT
@thiagoavadore#ITNEXT
@thiagoavadore#ITNEXT
BIG DATA ADMIN
SYSADMIN
DATA STEWARD
DATA
SCIENTIST
ML
ENGINEER
DATA ENGINEER
DATA FILL-THE-GAP
#ITNEXT @thiagoavadore
CLOUD! 

WOOHOOOOO!
@thiagoavadore#ITNEXT
DATA HAS GRAVITY.

STOP FIGHTING OVER IT.
#ITNEXT @thiagoavadore
CLOUD WARS!
@thiagoavadore#ITNEXT
#ITNEXT @thiagoavadore
Kinesis + Hadoop on EC2
#ITNEXT @thiagoavadore
Kinesis + Hadoop on EC2
#ITNEXT @thiagoavadore
Kinesis + Hadoop on EC2
#ITNEXT @thiagoavadore
Kinesis + Everlasting EMR
#ITNEXT @thiagoavadore
Kinesis + Everlasting EMR
#ITNEXT @thiagoavadore
Kinesis + Everlasting EMR
#ITNEXT @thiagoavadore
Kinesis + S3 + Redshift +
DynamoDB/HBase on EMR +
Ephemeral EMR’s
#ITNEXT @thiagoavadore
Kinesis + S3 + Redshift +
DynamoDB/HBase on EMR +
Ephemeral EMR’s
#ITNEXT @thiagoavadore
Kinesis + S3 + Redshift +
DynamoDB/HBase on EMR +
Ephemeral EMR’s
#ITNEXT @thiagoavadore
Kinesis + S3 + Redshift +
DynamoDB/HBase on EMR +
Ephemeral EMR’s
#ITNEXT @thiagoavadore
Kinesis + S3 + Redshift +
DynamoDB/HBase on EMR +
Ephemeral EMR’s
#ITNEXT @thiagoavadore
Kinesis + S3 + Redshift +
DynamoDB/HBase on EMR +
Ephemeral EMR’s
#ITNEXT @thiagoavadore
Kinesis + S3 + Redshift +
DynamoDB/HBase on EMR +
Ephemeral EMR’s
#ITNEXT @thiagoavadore
Honorable mentions
@thiagoavadore#ITNEXT
@thiagoavadore#ITNEXT
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
EventHubs + Hadoop on VM's
#ITNEXT @thiagoavadore
EventHubs + Hadoop on VM's
#ITNEXT @thiagoavadore
EventHubs + Hadoop on VM's
#ITNEXT @thiagoavadore
EventHub +
Everlasting HDInsight
#ITNEXT @thiagoavadore
EventHub +
Everlasting HDInsight
#ITNEXT @thiagoavadore
EventHub +
Everlasting HDInsight
#ITNEXT @thiagoavadore
EventHub +
Everlasting HDInsight
#ITNEXT @thiagoavadore
EventHub + Storage/DLS + DataWarehouse +
TableStore/HBase on HDInsight + Ephemeral
HDInsight
#ITNEXT @thiagoavadore
EventHub + Storage/DLS + DataWarehouse +
TableStore/HBase on HDInsight + Ephemeral
HDInsight
#ITNEXT @thiagoavadore
EventHub + Storage/DLS + DataWarehouse +
TableStore/HBase on HDInsight + Ephemeral
HDInsight
#ITNEXT @thiagoavadore
EventHub + Storage/DLS + DataWarehouse +
TableStore/HBase on HDInsight + Ephemeral
HDInsight
#ITNEXT @thiagoavadore
EventHub + Storage/DLS + DataWarehouse +
TableStore/HBase on HDInsight + Ephemeral
HDInsight
#ITNEXT @thiagoavadore
EventHub + Storage/DLS + DataWarehouse +
TableStore/HBase on HDInsight + Ephemeral
HDInsight
#ITNEXT @thiagoavadore
EventHub + Storage/DLS + DataWarehouse +
TableStore/HBase on HDInsight + Ephemeral
HDInsight
#ITNEXT @thiagoavadore
Honorable mentions
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
@thiagoavadore#ITNEXT
#ITNEXT @thiagoavadore
PubSub 

+ Hadoop on Compute Engines
#ITNEXT @thiagoavadore
PubSub 

+ Hadoop on Compute Engines
#ITNEXT @thiagoavadore
PubSub 

+ Hadoop on Compute Engines
#ITNEXT @thiagoavadore
PubSub + Everlasting
Dataproc
#ITNEXT @thiagoavadore
PubSub + Everlasting
Dataproc
#ITNEXT @thiagoavadore
PubSub + Everlasting
Dataproc
#ITNEXT @thiagoavadore
PubSub + Storage + BigQuery + BigTable/HBase
on DataProc + Ephemeral DataProc
#ITNEXT @thiagoavadore
PubSub + Storage + BigQuery + BigTable/HBase
on DataProc + Ephemeral DataProc
#ITNEXT @thiagoavadore
PubSub + Storage + BigQuery + BigTable/HBase
on DataProc + Ephemeral DataProc
#ITNEXT @thiagoavadore
PubSub + Storage + BigQuery + BigTable/HBase
on DataProc + Ephemeral DataProc
#ITNEXT @thiagoavadore
PubSub + Storage + BigQuery + BigTable/HBase
on DataProc + Ephemeral DataProc
#ITNEXT @thiagoavadore
PubSub + Storage + BigQuery + BigTable/HBase
on DataProc + Ephemeral DataProc
#ITNEXT @thiagoavadore
PubSub + Storage + BigQuery + BigTable/HBase
on DataProc + Ephemeral DataProc
#ITNEXT @thiagoavadore
Honorable mentions
@thiagoavadore#ITNEXT
@thiagoavadore#ITNEXT
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
Serverless
Event-driven
Stream
#ITNEXT @thiagoavadore
*All pics belong to the amazing JD Hancock - http://photos.jdhancock.com/
#ITNEXT @thiagoavadore
#ITNEXT @thiagoavadore
THANKS!!
GO TO ITNEXT.IO

Mais conteúdo relacionado

PDF
DevOpsDays Istanbul 2019 - AI with a devops mindset
PDF
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
PDF
AI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
PDF
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
PDF
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
PDF
devopsdays Warsaw 2018 - Chaos while deploying ML
PDF
devopsdays Riga 2018 - Chaos while deploying AI and making sure it doesn’t hu...
PPTX
devopsdays Amsterdam 2018 - Chaos while deploying AI and making sure it doesn...
DevOpsDays Istanbul 2019 - AI with a devops mindset
Codemotion Rome 2019: AI with a DevOps Mindset - Experimentation, sharing & e...
AI with a DevOps Mindset - Experimentation, sharing & easy ML deployment
Codemotion Milan 2018 - AI with a devops mindset: experimentation, sharing an...
Codemotion Berlin 2018 - AI with a devops mindset: experimentation, sharing a...
devopsdays Warsaw 2018 - Chaos while deploying ML
devopsdays Riga 2018 - Chaos while deploying AI and making sure it doesn’t hu...
devopsdays Amsterdam 2018 - Chaos while deploying AI and making sure it doesn...
Anúncio

ITNEXT Meetup - Native storage, ephemeral clusters and the pitfalls of building your data pipeline in a cloud world