SlideShare a Scribd company logo
Google App Engine
(Platform as a Service)
Boston Cloud Services Meetup
J Singh
January 14, 2014
PaaS Goal: Focus on Development, not Ops
• Virtual Raised Floor
– IDE above the floor
– Website for visibility
and control below
the floor
– Deployment System
( Tile Lifter)

© DataThinks 2013-14
2

2
IDE Above the Floor
• The programmer’s development environment
– Presentation layer: HTML, CSS, JavaScript
– Control layer: Web Server code
• Access to external APIs

– Data layer: Data Model
• Indexing advice

– Optionally, analytics

© DataThinks 2013-14
3

3
Ops below the floor
• Made visible through a web interface
–
–
–
–
–
–

Operating System
File System
User Authentication
Utilities (cron, etc.)
Logs
Database maintenance, backups, etc.

© DataThinks 2013-14
4

4
Deployment System
• Methods for continuous deployment
– Upload
– Version management

© DataThinks 2013-14
5

5
Google App Engine
• Strategic Technology Offering for Server-side applications
– Vehicle for introducing internal technology to the outside
developer community
– Not to be confused with Google Apps
– Frequent criticism:
• Single-source
• Except for AppScale (GAE code that runs on Amazon)

© DataThinks 2013-14
6

6
Google App Engine History
• Introduced in 2008
– Python, Google DataStore

• Now
– Languages
• Python, Go, PHP, Java
– And languages that compile to JVM byte codes

– Data Stores
• Google DataStore (NoSQL), CloudStore (Cloud intf to MySQL)

– Map Reduce

© DataThinks 2013-14
7

7
Sources
• Getting Started Instructions: (http://goo.gl/Wc4A9R)
• Map Reduce Instructions: (http://goo.gl/gzmj7T)
• Code: (http://goo.gl/SqmCKk) (a Github repository)
– Commit 0e24b6ad7: Guestbook application
– Commit 68f929415: Fetching from a Gutenberg.org URL
• Gets “Permission Denied” from gutenberg.org
• Change to read & parse pages from Wikipedia or another
source

– Commit 1740fedc6: Map Reduce changes

© DataThinks 2013-14
8

8
Guestbook Application
• Application Demo
(http://goo.gl/ItxjME)

• Code walk through:
– Dispatching
– Code
– Templates

– Write something in the
guestbook,
– Log in, write again,
– …

• Change page text

• Console walk through:

• Delete some guestbook
entries

– Dashboard
– DataStore
– Logs

© DataThinks 2013-14
9

9
Map Reduce Flow

© DataThinks 2013-14
10

10
Map Reduce Pipelines
• Map Reduce is rarely a singular operation
• Multiple Map Reduce operations are pipelined together
– Fan out, synchronization semantics

© DataThinks 2013-14
11

11
Map Reduce Application
• Application Demo
(http://goo.gl/KUPDc1)

• Code walk through:
– WordCountPipeline
– word_count_map
– word_count_reduce

• Where are the results?

© DataThinks 2013-14
12

12
Thank you
• J Singh
– Principal, DataThinks
• j.singh@datathinks.org

– Adj. Prof, WPI

© DataThinks 2013-14
13

13

More Related Content

PDF
Unified Data Access with Gimel
PDF
Spark Summit EU talk by Shaun Klopfenstein and Neelesh Shastry
PDF
Machine Learning for Any Size of Data, Any Type of Data
PDF
How to teach your data scientist to leverage an analytics cluster with Presto...
PPTX
Building Data Pipelines with Spark and StreamSets
PPTX
Taboola Road To Scale With Apache Spark
PPTX
Self-Service Analytics on Hadoop: Lessons Learned
PDF
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...
Unified Data Access with Gimel
Spark Summit EU talk by Shaun Klopfenstein and Neelesh Shastry
Machine Learning for Any Size of Data, Any Type of Data
How to teach your data scientist to leverage an analytics cluster with Presto...
Building Data Pipelines with Spark and StreamSets
Taboola Road To Scale With Apache Spark
Self-Service Analytics on Hadoop: Lessons Learned
Presto: Fast SQL-on-Anything (including Delta Lake, Snowflake, Elasticsearch ...

What's hot (20)

PDF
Speeding Up Atlas Deep Learning Platform with Alluxio + Fluid
PPTX
HBaseCon 2012 | Developing Real Time Analytics Applications Using HBase in th...
PDF
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
PPTX
Augmenting Mongo DB with treasure data
PPT
Google App Engine
PPTX
Best Practices for Supercharging Cloud Analytics on Amazon Redshift
PPTX
Presto query optimizer: pursuit of performance
PPTX
Disrupting Big Data with Apache Spark in the Cloud
PPTX
BTUG - Dec 2014 - Hybrid Connectivity Options
PPTX
Big Data at Pinterest - Presented by Qubole
PDF
Spark and Hadoop at Production Scale-(Anil Gadre, MapR)
PDF
Presto: Fast SQL on Everything
PPTX
Spark Magic Building and Deploying a High Scale Product in 4 Months
PDF
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Alluxio (formerly Tachyon)...
PDF
Building Robust Production Data Pipelines with Databricks Delta
PDF
Architecture at Scale
PDF
How Spark Fits into Baidu's Scale-(James Peng, Baidu)
PDF
High Performance Data Lake with Apache Hudi and Alluxio at T3Go
PDF
Accelerate Analytics and ML in the Hybrid Cloud Era
PDF
Cornami Accelerates Performance on SPARK: Spark Summit East talk by Paul Master
Speeding Up Atlas Deep Learning Platform with Alluxio + Fluid
HBaseCon 2012 | Developing Real Time Analytics Applications Using HBase in th...
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Augmenting Mongo DB with treasure data
Google App Engine
Best Practices for Supercharging Cloud Analytics on Amazon Redshift
Presto query optimizer: pursuit of performance
Disrupting Big Data with Apache Spark in the Cloud
BTUG - Dec 2014 - Hybrid Connectivity Options
Big Data at Pinterest - Presented by Qubole
Spark and Hadoop at Production Scale-(Anil Gadre, MapR)
Presto: Fast SQL on Everything
Spark Magic Building and Deploying a High Scale Product in 4 Months
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Alluxio (formerly Tachyon)...
Building Robust Production Data Pipelines with Databricks Delta
Architecture at Scale
How Spark Fits into Baidu's Scale-(James Peng, Baidu)
High Performance Data Lake with Apache Hudi and Alluxio at T3Go
Accelerate Analytics and ML in the Hybrid Cloud Era
Cornami Accelerates Performance on SPARK: Spark Summit East talk by Paul Master
Ad

Viewers also liked (10)

PDF
5. the grid implementing production grid
PDF
Platform as a service google app engine
PDF
Distributed Computing with Apache Hadoop: Technology Overview
PPTX
Unit i introduction to grid computing
PPT
Introduction to Google App Engine
PDF
Hadoop Architecture and HDFS
PDF
Google app engine
PDF
1. GRID COMPUTING
PDF
Hadoop Overview & Architecture
 
PPT
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)
5. the grid implementing production grid
Platform as a service google app engine
Distributed Computing with Apache Hadoop: Technology Overview
Unit i introduction to grid computing
Introduction to Google App Engine
Hadoop Architecture and HDFS
Google app engine
1. GRID COMPUTING
Hadoop Overview & Architecture
 
Hadoop architecture (Delhi Hadoop User Group Meetup 10 Sep 2011)
Ad

Similar to PaaS - google app engine (20)

PPTX
Google Cloud Platform, Compute Engine, and App Engine
PPTX
CloudPlatforms-Cloud PLatforms evaluation
PDF
Cc unit 5
PDF
Google App Engine
PPTX
Google Cloud Platform
PDF
Google Cloud Platform Update
PDF
Java Web Programming Using Cloud Platform: Module 10
KEY
Introduction to Google App Engine
PDF
App Engine overview (Android meetup 06-10)
PDF
Introduction to Google Cloud Platform Technologies
PDF
Javaedge 2010-cschalk
PDF
Java Web Programming on Google Cloud Platform [1/3] : Google App Engine
PDF
Google App Engine 7 9-14
PDF
Google Cloud for Data Crunchers - Strata Conf 2011
PPTX
File Repository on GAE
PPT
App_Engine_PPT..........................
PPT
App_Engine_PPT.ppt
PDF
GDD Brazil 2010 - What's new in Google App Engine and Google App Engine For B...
PDF
Introduction to Google's Cloud Technologies
PPTX
Introduction to Google App Engine
Google Cloud Platform, Compute Engine, and App Engine
CloudPlatforms-Cloud PLatforms evaluation
Cc unit 5
Google App Engine
Google Cloud Platform
Google Cloud Platform Update
Java Web Programming Using Cloud Platform: Module 10
Introduction to Google App Engine
App Engine overview (Android meetup 06-10)
Introduction to Google Cloud Platform Technologies
Javaedge 2010-cschalk
Java Web Programming on Google Cloud Platform [1/3] : Google App Engine
Google App Engine 7 9-14
Google Cloud for Data Crunchers - Strata Conf 2011
File Repository on GAE
App_Engine_PPT..........................
App_Engine_PPT.ppt
GDD Brazil 2010 - What's new in Google App Engine and Google App Engine For B...
Introduction to Google's Cloud Technologies
Introduction to Google App Engine

More from J Singh (20)

PDF
OpenLSH - a framework for locality sensitive hashing
PPTX
Designing analytics for big data
PDF
Open LSH - september 2014 update
PPTX
Mining of massive datasets using locality sensitive hashing (LSH)
PPTX
Data Analytic Technology Platforms: Options and Tradeoffs
PPTX
Facebook Analytics with Elastic Map/Reduce
PPTX
Big Data Laboratory
PPTX
The Hadoop Ecosystem
PPTX
Social Media Mining using GAE Map Reduce
PPTX
High Throughput Data Analysis
PPTX
NoSQL and MapReduce
PPTX
CS 542 -- Concurrency Control, Distributed Commit
PPTX
CS 542 -- Failure Recovery, Concurrency Control
PPTX
CS 542 -- Query Optimization
PPTX
CS 542 -- Query Execution
PPTX
CS 542 Putting it all together -- Storage Management
PPTX
CS 542 Parallel DBs, NoSQL, MapReduce
PPTX
CS 542 Database Index Structures
PPTX
CS 542 Controlling Database Integrity and Performance
PPTX
CS 542 Overview of query processing
OpenLSH - a framework for locality sensitive hashing
Designing analytics for big data
Open LSH - september 2014 update
Mining of massive datasets using locality sensitive hashing (LSH)
Data Analytic Technology Platforms: Options and Tradeoffs
Facebook Analytics with Elastic Map/Reduce
Big Data Laboratory
The Hadoop Ecosystem
Social Media Mining using GAE Map Reduce
High Throughput Data Analysis
NoSQL and MapReduce
CS 542 -- Concurrency Control, Distributed Commit
CS 542 -- Failure Recovery, Concurrency Control
CS 542 -- Query Optimization
CS 542 -- Query Execution
CS 542 Putting it all together -- Storage Management
CS 542 Parallel DBs, NoSQL, MapReduce
CS 542 Database Index Structures
CS 542 Controlling Database Integrity and Performance
CS 542 Overview of query processing

Recently uploaded (20)

PDF
Spectral efficient network and resource selection model in 5G networks
PDF
Electronic commerce courselecture one. Pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PPTX
Spectroscopy.pptx food analysis technology
PDF
Encapsulation theory and applications.pdf
PDF
KodekX | Application Modernization Development
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PDF
Network Security Unit 5.pdf for BCA BBA.
PPTX
sap open course for s4hana steps from ECC to s4
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
cuic standard and advanced reporting.pdf
PDF
Empathic Computing: Creating Shared Understanding
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
NewMind AI Weekly Chronicles - August'25 Week I
Spectral efficient network and resource selection model in 5G networks
Electronic commerce courselecture one. Pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Reach Out and Touch Someone: Haptics and Empathic Computing
Spectroscopy.pptx food analysis technology
Encapsulation theory and applications.pdf
KodekX | Application Modernization Development
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Network Security Unit 5.pdf for BCA BBA.
sap open course for s4hana steps from ECC to s4
Diabetes mellitus diagnosis method based random forest with bat algorithm
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
20250228 LYD VKU AI Blended-Learning.pptx
cuic standard and advanced reporting.pdf
Empathic Computing: Creating Shared Understanding
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Encapsulation_ Review paper, used for researhc scholars
NewMind AI Weekly Chronicles - August'25 Week I

PaaS - google app engine

  • 1. Google App Engine (Platform as a Service) Boston Cloud Services Meetup J Singh January 14, 2014
  • 2. PaaS Goal: Focus on Development, not Ops • Virtual Raised Floor – IDE above the floor – Website for visibility and control below the floor – Deployment System ( Tile Lifter) © DataThinks 2013-14 2 2
  • 3. IDE Above the Floor • The programmer’s development environment – Presentation layer: HTML, CSS, JavaScript – Control layer: Web Server code • Access to external APIs – Data layer: Data Model • Indexing advice – Optionally, analytics © DataThinks 2013-14 3 3
  • 4. Ops below the floor • Made visible through a web interface – – – – – – Operating System File System User Authentication Utilities (cron, etc.) Logs Database maintenance, backups, etc. © DataThinks 2013-14 4 4
  • 5. Deployment System • Methods for continuous deployment – Upload – Version management © DataThinks 2013-14 5 5
  • 6. Google App Engine • Strategic Technology Offering for Server-side applications – Vehicle for introducing internal technology to the outside developer community – Not to be confused with Google Apps – Frequent criticism: • Single-source • Except for AppScale (GAE code that runs on Amazon) © DataThinks 2013-14 6 6
  • 7. Google App Engine History • Introduced in 2008 – Python, Google DataStore • Now – Languages • Python, Go, PHP, Java – And languages that compile to JVM byte codes – Data Stores • Google DataStore (NoSQL), CloudStore (Cloud intf to MySQL) – Map Reduce © DataThinks 2013-14 7 7
  • 8. Sources • Getting Started Instructions: (http://goo.gl/Wc4A9R) • Map Reduce Instructions: (http://goo.gl/gzmj7T) • Code: (http://goo.gl/SqmCKk) (a Github repository) – Commit 0e24b6ad7: Guestbook application – Commit 68f929415: Fetching from a Gutenberg.org URL • Gets “Permission Denied” from gutenberg.org • Change to read & parse pages from Wikipedia or another source – Commit 1740fedc6: Map Reduce changes © DataThinks 2013-14 8 8
  • 9. Guestbook Application • Application Demo (http://goo.gl/ItxjME) • Code walk through: – Dispatching – Code – Templates – Write something in the guestbook, – Log in, write again, – … • Change page text • Console walk through: • Delete some guestbook entries – Dashboard – DataStore – Logs © DataThinks 2013-14 9 9
  • 10. Map Reduce Flow © DataThinks 2013-14 10 10
  • 11. Map Reduce Pipelines • Map Reduce is rarely a singular operation • Multiple Map Reduce operations are pipelined together – Fan out, synchronization semantics © DataThinks 2013-14 11 11
  • 12. Map Reduce Application • Application Demo (http://goo.gl/KUPDc1) • Code walk through: – WordCountPipeline – word_count_map – word_count_reduce • Where are the results? © DataThinks 2013-14 12 12
  • 13. Thank you • J Singh – Principal, DataThinks • j.singh@datathinks.org – Adj. Prof, WPI © DataThinks 2013-14 13 13