SlideShare a Scribd company logo
Data @ Streak
How we use the Google Cloud Platform

Aleem Mawani, aleem@streak.com, @aloo
What is Streak?
What do we use Google Cloud Platform for?

1. Serve our app and run business logic
AppEngine + Prediction API + Search API

2. Store and Analyze user data
AppEngine Datastore + Memcache + Cloud Storage + BigQuery

3. Record and analyze every UI action
BigQuery (Streaming + Bigquery Extension)

4. Record and analyze all server logs
BigQuery (Streaming + Bigquery Extension)

5. Record and analyze all client logs
BigQuery (Streaming + Bigquery Extension)
Streak + Google Cloud Platform
Streak + Google Cloud Platform
Streak on Google Cloud

App Engine

...

app
data

Datastore +
Memcache
Streak on Google Cloud

Logstore

lo

App Engine

dat
a

g

da

ta

log

BigQuery

...

app
data

Datastore +
Memcache
Streak on Google Cloud

Logstore

lo

App Engine

dat
a

g

da

ta

log

BigQuery

...

app
data

Datastore +
Memcache

app
data
Cloud
Storage
Prediction
API
Streak on Google Cloud

Logstore

lo

App Engine

dat
a

g

da

ta

log

BigQuery

...

app
data

Datastore +
Memcache

app
data
Cloud
Storage

ata

pd

ap
App Engine
Search API

Prediction
API
Streak on Google Cloud

Logstore

lo

App Engine

dat
a

g

da

ta

log

BigQuery

Cloud
Storage

ata

pd

Translation API &
Human Translators

app
data

ap

n

Datastore +
Memcache

i18

...

app
data

App Engine
Search API

Prediction
API
Streak on Google Cloud

t

ta
lo

g

ta

da

da

App Engine

dat
a

ata
nt d

en

log

event d
ata

eve

ev

Logstore

BigQuery

Cloud
Storage

ata

pd

Translation API &
Human Translators

app
data

ap

n

Datastore +
Memcache

i18

...

app
data

App Engine
Search API

Prediction
API
BigQuery Demo
Thanks!

Like big data? Streak.com is hiring!
jobs@streak.com
Summary

Benefits
● Move fast and build lean!
● Google quality infrastructure for everyone
● Ask critical business questions
Tradeoffs
● Need to move data incrementally
● Need to make your pipelines fault tolerant at scale
Tips
● Mirror your data in multiple systems, storage is cheap!
● Build systems incrementally

More Related Content

PDF
Using ap is to gather data
PDF
Getting your data in and out of elasticsearch: let me count the ways
PDF
Google Cloud Platform for the Enterprise
PPTX
Introduction to Google Cloud Platform
PDF
A Tour of Google Cloud Platform
PDF
Exploring the Google Analytics API
PDF
Big Data Driven At Eway
PPT
Google cloud platform
Using ap is to gather data
Getting your data in and out of elasticsearch: let me count the ways
Google Cloud Platform for the Enterprise
Introduction to Google Cloud Platform
A Tour of Google Cloud Platform
Exploring the Google Analytics API
Big Data Driven At Eway
Google cloud platform

Similar to Streak + Google Cloud Platform (20)

PPTX
File Repository on GAE
PDF
CodeCamp Iasi - Creating serverless data analytics system on GCP using BigQuery
PDF
Entrepreneurship Tips With HTML5 & App Engine Startup Weekend (June 2012)
PDF
#MBLTdev: Разработка backend для мобильного приложения с использованием Googl...
PDF
Scraper API To Acquire Real-Time Data Using Python.pdf
PPTX
Sumo Logic Webinar: Visibility into your Host Metrics
PDF
Introduction to Google Cloud Platform Technologies
PDF
Google Cloud Platform Update
PDF
ALT-F1.BE : The Accelerator (Google Cloud Platform)
PDF
Cloud computing overview & running your code on Google Cloud (Jun 2019)
PDF
Streamlining data analysis through environmental alerts how to integrate ambe...
PPTX
MongoDB World 2018: Ch-Ch-Ch-Ch-Changes: Taking Your Stitch Application to th...
PDF
Javaedge 2010-cschalk
PDF
End To End Business Intelligence On Google Cloud
PDF
Google Cloud Platform - Building a scalable mobile application
PDF
Google Cloud Platform - Building a scalable Mobile Application
PPTX
Automating some google things
PPTX
Introduction to Digital Analytics for Apps - Trusted Conf
PDF
AnDevCon - Tracking User Behavior Creatively
PPTX
Introduction Data Warehouse With BigQuery
File Repository on GAE
CodeCamp Iasi - Creating serverless data analytics system on GCP using BigQuery
Entrepreneurship Tips With HTML5 & App Engine Startup Weekend (June 2012)
#MBLTdev: Разработка backend для мобильного приложения с использованием Googl...
Scraper API To Acquire Real-Time Data Using Python.pdf
Sumo Logic Webinar: Visibility into your Host Metrics
Introduction to Google Cloud Platform Technologies
Google Cloud Platform Update
ALT-F1.BE : The Accelerator (Google Cloud Platform)
Cloud computing overview & running your code on Google Cloud (Jun 2019)
Streamlining data analysis through environmental alerts how to integrate ambe...
MongoDB World 2018: Ch-Ch-Ch-Ch-Changes: Taking Your Stitch Application to th...
Javaedge 2010-cschalk
End To End Business Intelligence On Google Cloud
Google Cloud Platform - Building a scalable mobile application
Google Cloud Platform - Building a scalable Mobile Application
Automating some google things
Introduction to Digital Analytics for Apps - Trusted Conf
AnDevCon - Tracking User Behavior Creatively
Introduction Data Warehouse With BigQuery
Ad

More from BigDataCloud (20)

PDF
Webinar - Comparative Analysis of Cloud based Machine Learning Platforms
PDF
Crime Analysis & Prediction System
PDF
REAL-TIME RECOMMENDATION SYSTEMS
PDF
Cloud Computing Services
PDF
Google Enterprise Cloud Platform - Resources & $2000 credit!
PDF
Big Data in the Cloud - Solutions & Apps
PDF
Big Data Analytics in Motorola on the Google Cloud Platform
PDF
Using Advanced Analyics to bring Business Value
PDF
Creating Business Value from Big Data, Analytics & Technology.
PDF
Deep Learning for NLP (without Magic) - Richard Socher and Christopher Manning
PPTX
Recommendation Engines - An Architectural Guide
PPTX
Why Hadoop is the New Infrastructure for the CMO?
PDF
Hadoop : A Foundation for Change - Milind Bhandarkar Chief Scientist, Pivotal
PPTX
Big Data Cloud Meetup - Jan 29 2013 - Mike Stonebraker & Scott Jarr of VoltDB
PPTX
Big Data Cloud Meetup - Jan 24 2013 - Zettaset
PDF
A Survey of Petabyte Scale Databases and Storage Systems Deployed at Facebook
PDF
What Does Big Data Mean and Who Will Win
PDF
Big Data Analytics in a Heterogeneous World - Joydeep Das of Sybase
PDF
BigDataCloud meetup Feb 16th - Microsoft's Saptak Sen's presentation
PPT
BigDataCloud Sept 8 2011 Meetup - Fail-Proofing Hadoop Clusters with Automati...
Webinar - Comparative Analysis of Cloud based Machine Learning Platforms
Crime Analysis & Prediction System
REAL-TIME RECOMMENDATION SYSTEMS
Cloud Computing Services
Google Enterprise Cloud Platform - Resources & $2000 credit!
Big Data in the Cloud - Solutions & Apps
Big Data Analytics in Motorola on the Google Cloud Platform
Using Advanced Analyics to bring Business Value
Creating Business Value from Big Data, Analytics & Technology.
Deep Learning for NLP (without Magic) - Richard Socher and Christopher Manning
Recommendation Engines - An Architectural Guide
Why Hadoop is the New Infrastructure for the CMO?
Hadoop : A Foundation for Change - Milind Bhandarkar Chief Scientist, Pivotal
Big Data Cloud Meetup - Jan 29 2013 - Mike Stonebraker & Scott Jarr of VoltDB
Big Data Cloud Meetup - Jan 24 2013 - Zettaset
A Survey of Petabyte Scale Databases and Storage Systems Deployed at Facebook
What Does Big Data Mean and Who Will Win
Big Data Analytics in a Heterogeneous World - Joydeep Das of Sybase
BigDataCloud meetup Feb 16th - Microsoft's Saptak Sen's presentation
BigDataCloud Sept 8 2011 Meetup - Fail-Proofing Hadoop Clusters with Automati...
Ad

Recently uploaded (20)

PDF
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
PDF
project resource management chapter-09.pdf
PDF
Encapsulation theory and applications.pdf
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
PPTX
Tartificialntelligence_presentation.pptx
PDF
A novel scalable deep ensemble learning framework for big data classification...
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Enhancing emotion recognition model for a student engagement use case through...
PDF
WOOl fibre morphology and structure.pdf for textiles
PPTX
TLE Review Electricity (Electricity).pptx
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
1 - Historical Antecedents, Social Consideration.pdf
PDF
Encapsulation_ Review paper, used for researhc scholars
PPTX
A Presentation on Touch Screen Technology
PPTX
SOPHOS-XG Firewall Administrator PPT.pptx
PDF
A comparative analysis of optical character recognition models for extracting...
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
project resource management chapter-09.pdf
Encapsulation theory and applications.pdf
Digital-Transformation-Roadmap-for-Companies.pptx
Unlocking AI with Model Context Protocol (MCP)
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
Tartificialntelligence_presentation.pptx
A novel scalable deep ensemble learning framework for big data classification...
Programs and apps: productivity, graphics, security and other tools
Enhancing emotion recognition model for a student engagement use case through...
WOOl fibre morphology and structure.pdf for textiles
TLE Review Electricity (Electricity).pptx
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
1 - Historical Antecedents, Social Consideration.pdf
Encapsulation_ Review paper, used for researhc scholars
A Presentation on Touch Screen Technology
SOPHOS-XG Firewall Administrator PPT.pptx
A comparative analysis of optical character recognition models for extracting...
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
Group 1 Presentation -Planning and Decision Making .pptx

Streak + Google Cloud Platform

  • 1. Data @ Streak How we use the Google Cloud Platform Aleem Mawani, aleem@streak.com, @aloo
  • 3. What do we use Google Cloud Platform for? 1. Serve our app and run business logic AppEngine + Prediction API + Search API 2. Store and Analyze user data AppEngine Datastore + Memcache + Cloud Storage + BigQuery 3. Record and analyze every UI action BigQuery (Streaming + Bigquery Extension) 4. Record and analyze all server logs BigQuery (Streaming + Bigquery Extension) 5. Record and analyze all client logs BigQuery (Streaming + Bigquery Extension)
  • 6. Streak on Google Cloud App Engine ... app data Datastore + Memcache
  • 7. Streak on Google Cloud Logstore lo App Engine dat a g da ta log BigQuery ... app data Datastore + Memcache
  • 8. Streak on Google Cloud Logstore lo App Engine dat a g da ta log BigQuery ... app data Datastore + Memcache app data Cloud Storage Prediction API
  • 9. Streak on Google Cloud Logstore lo App Engine dat a g da ta log BigQuery ... app data Datastore + Memcache app data Cloud Storage ata pd ap App Engine Search API Prediction API
  • 10. Streak on Google Cloud Logstore lo App Engine dat a g da ta log BigQuery Cloud Storage ata pd Translation API & Human Translators app data ap n Datastore + Memcache i18 ... app data App Engine Search API Prediction API
  • 11. Streak on Google Cloud t ta lo g ta da da App Engine dat a ata nt d en log event d ata eve ev Logstore BigQuery Cloud Storage ata pd Translation API & Human Translators app data ap n Datastore + Memcache i18 ... app data App Engine Search API Prediction API
  • 13. Thanks! Like big data? Streak.com is hiring! jobs@streak.com
  • 14. Summary Benefits ● Move fast and build lean! ● Google quality infrastructure for everyone ● Ask critical business questions Tradeoffs ● Need to move data incrementally ● Need to make your pipelines fault tolerant at scale Tips ● Mirror your data in multiple systems, storage is cheap! ● Build systems incrementally