Data driven culture & infrastructure from the ground up
Download as PPTX, PDF0 likes198 views
This document discusses building a data-driven culture and infrastructure from the ground up. It describes an R&D and IT consultancy firm that works with Silicon Valley startups. The firm has a hardcore big data analytics team in Kazan. It outlines how different roles on a data-driven team, like product managers, engineers, and data scientists, interact with data and metrics. It also presents different levels of infrastructure for handling and analyzing data, from micro to big systems, involving technologies like Kafka, Spark, Redshift, and Hadoop.
Data driven culture & infrastructure from the ground up
- 1. Data-driven culture & infrastructure from the ground up January 2017
- 2. Data-driven culture and infrastructure from the ground up Agenda ● R&D and IT consultancy firm ● Clients - Silicon Valley Startups from Round A to Round D ● Hardcore Big Data Analytics team in Kazan
- 3. Data-driven culture and infrastructure from the ground up January 2017 People driven
- 4. Data-driven culture and infrastructure from the ground up Excel driven January 2017
- 5. Data-driven culture and infrastructure from the ground up BI driven January 2017
- 6. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Manager Owners Engineer UI Data Scientist Sales/Marketing DevOps Customer
- 7. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports
- 8. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Engineer Alerts, Errors Metrics
- 9. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Engineer Alerts, Errors Metrics UX Usage Patterns A/B tests
- 10. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Engineer Alerts, Errors Metrics Data Scientist Scripts Models UX Usage Patterns A/B tests
- 11. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Owners Revenue trends Prescriptions Insights Engineer Alerts, Errors Metrics Data Scientist Scripts Models UX Usage Patterns A/B tests
- 12. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Owners Revenue trends Prescriptions Insights Engineer Alerts, Errors Metrics Data Scientist Scripts Models Marketing BI tools A/B test UX Usage Patterns A/B tests
- 13. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Owners Revenue trends Prescriptions Insights Engineer Alerts, Errors Metrics DevOps Alerts, Errors Metrics Monitoring Data Scientist Scripts Models Marketing BI tools A/B test UX Usage Patterns A/B tests
- 14. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Customer Dashboards Personalization Insights Reports Owners Revenue trends Prescriptions Insights Engineer Alerts, Errors Metrics DevOps Alerts, Errors Metrics Monitoring Data Scientist Scripts Models Marketing BI tools A/B test UX Usage Patterns A/B tests
- 15. Data-driven culture and infrastructure from the ground up Data Driven Team January 2017 Product Metrics Ad-hoc reports Reports Customer Dashboards Personalization Insights Reports Owners Revenue trends Prescriptions Insights Engineer Alerts, Errors Metrics DevOps Alerts, Errors Metrics Monitoring Data Scientist Scripts Models Marketing BI tools A/B test UX Usage Patterns A/B tests
- 16. Data-driven culture and infrastructure from the ground up ● Product Managers - SQL ● Engineers - Production, Product, Metrics, SOFT SKILLS ● UX - Marketing, Analytics ● Sales Engineer ● Business - BI, Reports, Metrics January 2017 New Cross Skills - Evangelism
- 17. Data-driven culture and infrastructure from the ground up Infrastructure
- 18. Data-driven culture and infrastructure from the ground up ● You could not just BUY it ● It’s not a Data Warehouse ● It’s not easy at all January 2017 Myths
- 19. Data-driven culture and infrastructure from the ground up High Level January 2017 App Servers Production database Logs Storage BI Prescriptions Application Data Infrastructure Visualisation
- 20. Data-driven culture and infrastructure from the ground up Micro January 2017 App Servers Production database Logs agg/ Papertrail Keen.io Clicdata Datadog Prometheus Application Data Infrastructure Visualisation GA
- 21. Data-driven culture and infrastructure from the ground up Small January 2017 App Servers Production database Log agg/ Papertrail S3 Clicdata Interana Looker Datadog Prometheus Application Data Infrastructure Visualisation Slave replica
- 22. Data-driven culture and infrastructure from the ground up Medium January 2017 App Servers Prod Log stream S3 Clicdata Interana Looker Datadog Prometheus Application Data Infrastructure Visualisation Slave replica ETL RedShift Ad-hoc SQL
- 23. Data-driven culture and infrastructure from the ground up Medium January 2017 App Servers Prod AWS Kinesis S3 Clicdata Interana Looker Datadog Application Data Infrastructure Visualisation Slave replica AWS Lambda RedShift Ad-hoc SQL
- 24. Data-driven culture and infrastructure from the ground up Medium January 2017 App Servers Prod Kafka S3 Clicdata Interana Looker Datadog Prometheus Application Data Infrastructure Visualisation Slave replica Spark Streaming RedShift Ad-hoc SQL
- 25. Data-driven culture and infrastructure from the ground up Big January 2017 App Servers Prod Tableau Clicdata Interana Looker Application Data Infrastructure Visualisation Slave replica ETL/Spark/Streaming RedShift Ad-hoc SQL Hadoop/Parquet Impala Spark SQL Data Science
- 26. www.squadex.com 125 University Avenue, Suite 290, Palo Alto, California, 94301 Questions, details? We would be happy to answer!