Google Dev Fest Presentation

Google App Engine &
Google Web Toolkit
our experience
developing
TouchActive

Andrew Mackenzie
Daniel Ruiz
BCNTouch
http://ta.bcntouch.com

The purpose of this talk

● Share our experiences using GAE and GWT

● Help guide you if you are considering similar
technologies

● Provide ideas if you are already developing
with these technologies

● Provide feedback to Google on things to
maintain, improve and renew/drop

Contents
● Solution
Our Project ● PaaS choice
● GAE overview
● GWT overview
GAE + GWT ● GAE + GWT debugging
● TouchActive
implementation
Implementation ● Addit
● Optimization tools
● Phonegap experiment
Summary ● Avoiding lock-in
● What would we do
differently
Feedback ● Feedback to Google

TouchActive Platform
- what we built
Brand

A platform to put Using tagged
consumers and products that the
brands in touch with consumers were
each other interested in

Consumer

Mobile apps to read tagged products
- NFC, QR, barcode
Scalable store in the
cloud with product
Cloud service and data and
data storage synchronization of
user data

Mobile apps read
products tagged
with various
technologies

Get product
information, help
remember, organize
in lists, etc.

Focus on easy and intuitive interaction with
no data entry. NFC the best.

Get information about products
from tagged objects

Management console for brand
- multi-tenant, groups, roles

Access control on
changes

Cloud service and
data storage

Products findable in
open queries from
mobile devices

Consumer Web Interface
- Lists, Products, History, etc
Mobile web version for mobile
platforms without a native app

Addit "Web Clipper" to capture
products from web pages

User data synchronized
with native mobile app Cloud service and
data storage

Social networks, Identity and
Sharing
Leverage existing
user identities for
easy sign-up
(web and mobile)
- OpenId and
OAuth

Sharing of Products,
Lists via:

● Social Networks
● EMail
● NFC Tags
● NFC peer-to-peer
● QR codes
● Bump

TouchActive Solution Overview
- what we built

TouchActive Solution Overview
- what we built

Desktop and Mobile Identity
Management consumer web Providers
console
Addit Sharing (web
and mobile)
over net and
Cloud backend
face-to-face

Mobile Apps

Tagged
products

Context for our choices
2+ years ago
● Start Small
○ pre-start-up: build prototype
○ uncertain financing (probably small)
○ small team of engineers

● Think big, or the possibility of big!

● Uncertainties
○ Mobile platforms (especially NFC)
○ Cloud computing platforms

Cooling

Power / UPS

Security

Buildings

LAMP stack on owned Amazon Web Services Google App Engine
or hosted HW (AWS) (GAE)

small start-up, low Newer options:
capital investment. ● OpenStack (Open SW, multiple service
Pay as you go, don't providers)
buy servers. ● Windows Azure
● Amazon features for auto scaling, etc.
unknown/unexpected
demand
we
don't spend people time and money on managing
servers, runtimes, etc. Focus on the application. chose
GAE
Cooling

Power / UPS

Security But we didn't want to get "locked-in"
Buildings to a single service provider

Applications developed
with GWT

AdWords AdSense Orkut

Wave Health Blogger

And many more, see:
http://gwtreferencelist.appspot.com
https://developers.google.com/web-toolkit/developer_spotlight
http://gwtgallery.appspot.com/
http://www.quora.com/What-web-applications-use-Google-Web-Toolkit-(GWT)

Google Web Toolkit
Complex UI

Complex UI with
graphical Data
representation

Google App Engine Features
● Free developer / Pay-as-you go costs
● Platform not Infrastructure
○ No need to architect the SW stack for scalability
○ No need to manage the SW stack
○ No machine administration
○ Reasonable set of platform services
● Scalability
○ Instances created automatically based on demand
○ Scalable, distributed, datastore
○ Content Distribution Network (CDN)
○ High-performance Image service

Google App Engine
Tools and Languages
Language choices at the time: Java and Python
○ Now "GO" is also available now
We chose Java
● Developing J2ME and Android mobile apps in Java
at the same time
Eclipse IDE
○ Local development server
○ Java debugging of server code
Appstats for server performance analysis (RPC)
Management dashboard
Command line tools and ANT tasks for automation

Google App Engine
Datastore Datastore

Megatable

BigTable

GFS2

Advantages Drawbacks
● Scalable ● Pay-per-use
● Time vary with result set ● NoSql & pre-built
size, not search set size indexes
● Compatible with existing ○ No JOIN operations
frameworks ○ No inequality filters on more
○ Java Data Objects (JDO) than one "field"
○ Java Persistence API (JPA) ○ No filtering of data based on
● Distributed, replicated subqueries
across data centers ○ No Grouping of results
● Write rate limited

Google App Engine
Platform Services
Standard Implementations Non-Standard
● Mail (JMail) ● Cron Tasks
● Datastore (JPA/JDO) ● Task Queues
● MemCache (JSR107) ● Back-ends
● Logging (Java Logging) ● Users (Identity)
● Blob Store
● Image Service
● Chat (XMPP/Jabber)
with Presence
● Channels API (Push)

Web Client Development
Google Web Toolkit
● Develop Javascript AJAX client with Java
○ Strongly, statically, typed language.
○ Many errors caught at compile time
○ Full IDE support (completion, etc)
○ Internationalization (i18n) built-in
○ Generates optimized JavaScript, client bundles, code splitting, etc.
○ Hides browser differences
■ Compiles a "permutation" for each language and browser you
support

● Integrated Server and Client development project
○ Java across Server and Client
○ Shared classes between Server and Client
○ "Transparent" RPCs between server and client (takes care of

Google Web Toolkit
● Deferred binding with "properties"
○ Used for loading script variants based on browser, language, form
factor, etc

● Reasonable Widget set
○ Has improved a lot
○ Other widget sets available on top (GXT from Sencha, SmartGWT,
Vadim, etc)

● "DevMode"
○ Java debug experience in IDE for client side code
○ Combined Client/Server/RPC debug experience
○ No-compile, fast turn-around, hot code replace
○ Memory intensive! :-(

Google Web Toolkit
image
Encourages reusable
modular code development
● Custom widgets
● Code in defined "modules"
(server/client/shared code,
RPCs, html, css, images, etc)
● Bundle modules into GWT
libraries (.jar)

Google Web Toolkit
Ease of implementing
Design Patterns
● Model-View-Presenter with
late-binding of views
● Same Presenter, different
Views
● Detect Form Factor from
UA and serve the most
appropriate view at run-
time

Google Web Toolkit
Late-Binding of views with ClientFactory

TouchActive.java

TouchActive.gwt.
xml

Google Web Toolkit
Late-Binding of views with ClientFactory

TouchActive.java

Desktop Mobile

Google Web Toolkit
MVP, Activities & Places

Event-Bus across client code for
subscribe/broadcast event
programming

Application Activity and Place
management via browser history
History Token

Google Web Toolkit
UiBinder (XML) for declarative layout

With standard or custom widgets

GAE/GWT Debug experience:
An example

Click Event on
Facebook sign
in button

Client debugging

Server debugging

Our implementation of TouchActive
with GAE + GWT

● Authentication / Identity management
● Architecture overview
● Data Persistence Layer
● Platform API
● Portability
● CMS Integration
● Addit

Authentication:
Own, portable, Identity management
● Created own User Object and related to standard
external accounts
○ OAuth (FB, Twitter, LikedIn)
○ OpenID (Google, Yahoo, MySpace, ...)
○ Google Apps domains for company users
● Username + Password also
● Avoid forcing users having to have a Google account
● Use OAuth & OpenId on mobile apps transparently for
account creation and sign-up

Avoided lock-in, didn't use Google App Engine Users
Service

Architecture Overview
Request
● Headers, cookies, ...

Servlet Filters

Request App Logic
Context

Platform API

Data Persistence Layer Platform Specific Services

Servlet Filters
● Flexible, just map filters to servlets with a config file
using paths
○ /api
○ /rpc
○ /cms
○ /static
● Keeps servlet code clean and modular
● Used for:
○ Language selection
○ Desktop/Mobile pages while maintaining constant URL
○ Authentication + assignment user to their Multi-Tenancy namespace
○ Query logging for analytics

Servlet Filters:
EntryPoint Filter (The Special One)
Request
● Headers, cookies, ...

EntryPoint Filter Servlet Filters

Request
Context

● Different types of EntryPoint filters that set different permissions and tokens
○ Initializes the Request Context used in your app and DataManager
○ Use of Thread variables to make available to all code but multi-threaded
● Authenticates the user and sets the correct "Namespace" in the request context
● Defined the appropriate filter for each request based on Access Control rules in
web.xml

Data Persistence Layer
Request
Context
DataManager I/F to rest of app

Data Manager

JDO portable library across multiple
datastore implementations

Google Datastore MySQL

Google AppEngine AWS MySQL Server

Data Persistence Layer
DataManager Interface DataManager I/F to rest of app

● Created an API to manage data
objects for reading and writing Data Manager
● DataManager layer is used by all
app, app is kept 100% portable
JDO
Data Objects
● ACL, Protected, Global, etc...
● Enforce visibility and Google Datastore MySQL
permissions according to filters
used on request and object type
MySQL
Multi-Tenancy Google AppEngine AWS
Server

● Create data areas in the same
datastore
● Avoided using Google Datastore
Namespaces

Platform Specific Services
App Logic

Defined a Platform API for platform dependant services

GAE Implementation Other Implementation
Cron Tasks Cron Tasks

Task Queues Task Queues

Blobstore Blobstore

Image Service Image Service
Custom build process
Chat (XMPP) Chat (XMPP)
outside IDE
● ANT tasks to build, deploy, test, Channels (Push) Channels (Push)
release
● Run manually from Command
Line or scripts / cron
● Significant effort

Additional Need:
Web content beyond the app
Editable by non-programmers
No need for application release on each change
Localized to multiple languages

Use a CMS
GWT

CMS Content

Integrating a CMS
Chose to serve content, scripts and RPCs from same domain
○ Deploy CMS to same GAE "appID" as our app
○ This could maybe be avoided with subdomains...

Chose Vosao GAE/J based CMS
○ A bit immature at the time....still < 1.0
○ Uses Apache Velocity (Java based) scripting language
○ Java plugin can integrate tightly with your app and call app code directly
from within CMS web pages
○ Has locked us in to GAE at CMS level, but our code is still portable
○ Manual integration of config files, with help from Vosao team it could be
automated

Internationalization (i18N)
● Client GWT UI
○ Built-in i18N support in GWT, strings in property files
○ Permutation of script compiled for each language and correct
language loaded at run time

● Server messages
○ We have avoided as much as possible
○ Java Message Bundle + Properties files

● CMS content
○ Core feature of CMS. Individual strings or entire pages can be
localized with common template

Other features implemented
● Chat functionality with server (XMPP)
○ used for testing at the moment

● "Presence" detection of users (XMPP)

● Push messages to browser using the GAE Channel API
○ GWT Client JSNI Wrapper and EventBus connection

● "Addit" Bookmarklet and "web-clipper"

Addit -
Objectives
● Help users create products in TouchActive
by capturing product from any web page
● Reduce need for data entry by capturing as
much data as possible automatically
○ Use eCommerce/Search meta data when available
● Relatively simple and easy to use
● Broad usage
○ Support all main web browsers
○ Support as many web sites as possible
○ Desktop, Tablet and Mobile versions

Product meta-data
meta and micro-formats used by eCommerce
sites to provide data to search engines or
crawlers:
Various Standards Exist:

● Facebook
○ OpenGraph "product"
● Schema.org
● GoodRelations
● GoogleProducts
○ Google Merchant Center /
Google Product Search
● hProduct

Addit Installation

Drag and drop bookmarklet
from web-page to
bookmarks bar or folder

Install extension directly
● From Chrome Web Store
● Chrome, Safari, Firefox from web page

User clicks either the
Addit Use Bookmarklet or the Extension
button (FF, Safari, Chrome)

Addit Detect
language from
Load localized script from
cloud server and injected
Script loading page contents into the page

Keeping script out of
the bookmarklet or
extention allows us to
update the script
continually, without any
re-install or
redistribution to users

Script NOT from Same-
Origin as page and
security concerns and
barriers to overcome

Addit
Fetch user lists and detect currency

Fetches users lists from
their TouchActive
account using a Cross-
Domain "JSONP" RPC
call

Select default currency
based on page
language, page
contents and user
settings retrieved with
lists

Addit
product meta-data parsing

Parse DOM and detect
micro-format product
markup elements.
Prepares products for
user automatically

If multiple products
detected, a simple
product selector UI is
shown initially

If no product detected,
uses page-level markup
(meta, OpenGraph) or
general page
information (Title, etc)

Addit
Personalize and save to account

Users take the suggested
texts, or customize them.
Select image, price, etc.

Select one of their own lists
they would like to add the
product to

Add a personal note

... and "Addit" to their
TouchActive account

Will be automatically synced with their mobile app

Addit
Different layouts for different form factors
Desktop Mobile
Wider, no vertical scrolling, smaller mouse targets Narrower (adjust to Viewport),
vertical scrolling, larger touch targets

All Developed in GWT
● GWT XSI Cross-Site linker for CORS script loading
● Wrote own JSONP library for Cross-Site RPC
○ in Java with some JSNI
● GWT "late run-time binding" of scripts for Localization
and Form Factor permutations using custom "Property
Providers"
○ Detect "best guess" language from host page using multiple methods
(poor standards implementations)
○ Detect form factor using a combination of User Agent and display size
● Avoid older Layout Widgets that compile down to HTML
Tables (Box Model and CSS problems...)
● Fast product meta-data parsing from DOM using
"querySelectorAll()" and CSS3 selectors

Difficulties - mainly due to injecting a
script into a page you don't control

● Unknown, uncontrolled CSS / DOCTYPE / QuirksMode
○ Can't just do a "CSS Reset"
○ CSS properties influence complex layouts
○ DOCTYPES affect browser behavior (a nightmare on IE 7/8/9)
● "Difficult" sites
○ Don't follow W3C standard for language selection
○ iFrames/Frames/Frameset based
○ Flash sites!
○ Broken image links
● Different implementations of product markup standards
○ Especially prices "," vs "." EUR vs € before/after/middle (1€25)
● JavaScript image galleries and tricks
○ Images loaded but not visible
○ Images being pre-cached (not always in DOM as image)

IE and DOCTYPE and Quirks
For masochists only!
http://blogs.msdn.com/b/askie/archive/2009/03/23/understanding-compatibility-modes-in-internet-
explorer-8.aspx

Phonegap Experiment

Q Can you take a GWT compiled app (HTML, CSS, JS,
Images etc) and create a "native" app for multiple
platforms?

A Kind-of...
● GWT RPCs from Phonegap client to server!
○ Must be able to go cross-domain
○ We wrapped the RPC class so that it remaps the RPC endpoint to
default server when running "offline"
○ Version problems expected with serialization in the future!!
● Phonegap has problems with relative file paths
○ Processed GWT comiled output to use absolute paths
● Prototype produced from existing GWT app
○ Automated with ANT tasks

Optimization Tools Available
Chrome Developer Tools "Audit"
PageSpeed
AppStats for server GAE internal RPC
SpeedTracer
SpeedTracer with AppStats

GAE AppStats
understand AppEngine RPC usage

AppStats
drill-down to calls, duration and latency

SpeedTracer
Understanding client side performance

SpeedTracer
Network resource usage from client

Resource Loads
● HTML
● CSS
● JavaScript

SpeedTracer (client) integrated with
AppStats (server)

RPC request to server

Server side trace and timing
from AppStats on server

What's new in GWT2.5
● Super Dev Mode (experimental)
○ Replaces DevMode, uses "SourceMaps" in Chrome
to enable debugging in language JS was written in
(Java, Dart...)
● New compiler optimizations
○ Smaller, faster JavaScript code
○ With slightly longer compile times
● Elemental
○ Lightweight JavaScript library
● Updated ARIA support
● UIBinder enhancements
● Validation enhancements

GWT Compiled code
size reductions

GWT 2.4

100%

GWT 2.5

93%

GWT 2.5
Closure Compiler
87%

Did we avoid provider "Lock-in"?
Portable data layer built on JDO
● Maybe not the most optimal when running on GAE
● Have to work with limitations of JDO/GAE subset
● Over time, actually moving the data becomes a problem
Platform API has kept rest of code portable
● needs work to get running on additional platforms
GAE based CMS has increased it
● outside of the code, but still important for the project
● decouple by moving to multiple domain approach

Things we would do differently if we
started again

New competitors to GAE ● New AWS services bring it
closer to PaaS
● OpenStack from multiple
Analytics providers
● Analyze more closely and
prototype if time permits
Phonegap app

API and directory
structure

CMS

started again

New competitors to GAE

Analytics ● Avoid programming a lot of
that, and falling short
Phonegap app ● Expose analytics data and
use external tools
● Give business users top-
API and directory class tools with flexibility of
structure
analysis, drill-down etc
CMS

started again


Analytics

Phonegap app ● Design and structure web-app
for that from day one

API and directory
structure

CMS

started again


Analytics

Phonegap app
● Assume you will have to
API and directory structure version your external API
● Design app structure early on
○ statics, resources, API,
CMS CMS, RPC,...

started again


Analytics

Phonegap app

API and directory
● Explore using other CMS
structure
outside of GAE
● Sub-domains for "Single-
CMS Origin" content, RPCs

Possible Future Work
● Optimize
○ Using tools to focus our efforts
○ More use of memcache for DS accesses
● Explore use JDO3 (Polymorphism)
● Prototype alternative CMS
○ Run on sub-domain in parallel
● Write platform code for other platforms
○ Test on AWS or OpenStack?
● Phonegap app for other platforms
○ Blackberry, Windows Phone...
● Investigate Continuous Integration Support
○ Jenkins on GAE

GAE + GWT
The Good
● Scalable, without knowing about or managing the stack
● Free for developers, our costs have been pretty low
● Java across mobile, backend and client
● Programming large app "all together" as client-server
○ But avoid that permitting spaghetti, structure APIs and code
○ Hierarchy of reusable modules of client, server and shared code
○ Drop-in GTW libraries with client, server and shared code
● Excellent debug experience when it works!
○ including hot code replacement (server + client)
● Model-View-Presenter architecture combined with late-
binding
○ extend your app to mobile, tablet views more easily
● UI Binder for declarative UI layout, code for behavior

GAE + GWT
The Bad
DevMode problems
● Lack of maintenance on browser plug-ins
● Memory intensive

You have to put in effort to avoid "lock-in"

JDO
● Some people reporting long start-up times (meta-data related)
○ Has worked pretty well for us
● Some features missing: e.g. No Async writes

Feedback for Google
App Engine
● Don't introduce features or ways to use features that
increase lock-in.
○ If you do, we will go to "standard" stacks on AWS or OpenStack...
○ e.g. threads, etc

● More transparency on Billing and scheduler behaviour

● Need better attention to developers in forums
○ Paying customers!
○ Especially around scheduling and billing - invest people hours in this

● Channels API
○ Support multiple listeners per page and broadcast
○ Google supported JSNI GWT library

Feedback for Google
GWT Debugging
Don't break the debug experience!

● Combined server/RPC/client debug experience in a
single IDE is fantastic and unique (?) in developing an
AJAX client - cloud server application
○ Make it better! More maintenance of plug-in
○ Support at least on Chrome / WebKit browser
○ Less of a memory hog

● GWT 2.5 SuperDevMode does not replace it IMHO
○ list of limitations of seems to outweigh the advantages
○ main advantages (plug-in maintenance) are for Google

Feedback for Google
GWT - others
● Allow client-only code and server-only code in same file
○ E.g. Via @annotations
○ Avoid having to complicate the class hierarchy to
have shared/server-only/client-only code for a class

● Java Calendar emulation in GWT and allow for
shared-code date handling

Thank You!
Questions?
Andrew Mackenzie
andrew@mackenzie-serres.net
andrew.mackenzie@bcntouch.com

http://www.linkedin.com/in/andrewdmackenzie

Daniel Ruiz Giménez
druizgimenez@gmail.com
daniel.ruiz@bcntouch.com
http://www.linkedin.com/in/danielruizgimenez

http://www.slideshare.
net/andrewdmackenzie/developing-
touchactive-with-gae-and-gwt-google-
50 devfest-barcelona-2012

Google Dev Fest Presentation

More Related Content

Similar to Google Dev Fest Presentation (20)

Google Dev Fest Presentation