Designing the Call of Cthulhu app with Google App Engine

Designing the Call of Cthulhu app with Google App Engine Presented by Chris Bunch at UCSB CS 189A / 172 : Capstone Project / Software Engineering February 10, 2010 http://cs.ucsb.edu/~cgb

Motivation I am what you would call a “nerd” Plays nerdy board games Would like to upload stats from games and analyze them Could do other cool things as well

Arkham Horror Lots of variables that can be analyzed Co-op game: all v. the board Mix and match expansions Many different winning / losing conditions But fundamentally boils down to: Save the world from total destruction

App Requirements Must be able to upload game data Should be able to predict a win or loss given some preliminary info Can experiment with prediction algorithms Should be able to summarize game data

App Requirements Allows users to log in to upload their data Allows users to share data with each other Handles users in a sane fashion Harder than it sounds Display a top score list for users to view Game provides scoring metric

Enter the Cloud Why go the cloud route? Simple answer: You likely don’t have your own hardware to host this app on, so just use somebody else’s But cloud computing is confusing and not well defined yet, so...

Defining Cloud Computing Three layers, each building on the last Infrastructure: Programmer gets a virtual machine and / or reliable storage Most control of environment Must scale manually Amazon EC2, Eucalyptus

Defining Cloud Computing Three layers, each building on the last Platform: Programmer gets a scalable API Less control of environment Scaling done automatically Google App Engine, Microsoft Azure

Defining Cloud Computing Three layers, each building on the last Software: Programmer gets to “skin” an existing app No control of environment Scaling done automatically SalesForce, GMail

My Choice Clearly I want the platform! Platform: Scalable API = great! Less control of environment = ok Programability depends on APIs offered Have personal exp. w/ Google App Engine

What is Google App Engine? Scalable web framework first offered in 2008 Users program apps in: Python Java Unofficially: JRuby, Groovy, JavaScript

Sandboxed Environment Requests limited to 30 seconds Only white-listed libraries can be used Anything non-trivial is quota-ed No reading / writing to the filesystem Web communication over HTTP(S) only

Datastore API A giant hash table Primitives: Get, put, delete - same as regular HT Query, count - new operations Transactions supported (row-level only)

Datastore API GQL queries offered - subset of SQL Select statement, no JOINs Tip: Queries are the most expensive operation, use sparingly If possible, perform it asynchronously

Memcache API Equivalent to memcached Provides access to low latency LRU cache Roughly 100x faster than Datastore But unreliable - remember it’s a cache! Exposes get, set, delete operations Exposes atomic increment / decrement

URL Fetch API Equivalent to curl Used to grab web content from other sites Only allows for HTTP(S) data to be read Can also be done asynchronously Typically grabs images or mashup data

Mail API Equivalent to sendmail Can send or receive mail asynchronously Can also add attachments Not all types allowed - no .doc, .xls, or .ppt in Python SDK

XMPP API Equivalent to jabber Send and receive instant messages to appname.appspotchat.com App can’t join group chat yet Use as a notifier or alternate UI to app

Images API Equivalent to python-imaging Exposes image rotation, flipping, cropping, and histogram Image is stored internally as a blob, then retrieved and manipulated

Users API Equivalent to Google Accounts Provides for simple authentication Allows app to force login or admin on certain pages Also exposes URL to optionally login to

Blobstore API Allows for large file uploads (<50 MB) Users can upload videos, datasets, or large music files for later retrieval Upload must be done via form post

Cron API Equivalent to cron User specifies how often a web page should be accessed (e.g., daily, every Monday) Task is then run in the background Useful for running expensive data analysis

Task Queue API Spawns a new thread in the background But still must visit your app Configuration file specifies queues Useful for short to medium length tasks

To summarize: Many, many APIs available for what we need to do! Use Datastore to store / retrieve user data Use Users to authenticate users Gets around a lot of boilerplate authentication code

To summarize: Use Mail to e-mail users if their score is beat Use XMPP to allow users to add new data Use Cron to run compute-intensive code in the background

To summarize: Think about paid functionality: Maybe use Blobstore to allow users to upload videos explaining gameplay Handle greater traffic if site becomes popular

Let’s Get to Business Data Modeling Upload Stats Page Game Stats Page Game Prediction Page

Data Modeling class Game(db.Model): player = db.UserProperty() goo = db.StringProperty() comments = dbStringProperty(multiline=True) date = db.DateTimeProperty(auto_now_add=True)

Data Modeling num_of_players = db.IntegerProperty() won_game = db.BooleanProperty() doom_counters = db.IntegerProperty() terror_level = db.IntegerProperty() expansions_used = dbListProperty(str)

Uploading Stats class AddGames(webapp.RequestHandler): def get(self): user = users.get_current_user() if user: # means they’re logged in # write html form for getting data else: self.redirect(users.create_login_url(self.request.uri))

Uploading Stats def post(self): new_goo = self.request.get(‘goo’) # validate goo, make sure data type is right game = Game() game.goo = new_goo game.put()

Game Stats games = db.GqlQuery (“SELECT * FROM Game ORDER BY date desc”) for game in games: # find the goo w/ highest player win percentage # can also just display the info here

Game Prediction def get(self): # user specifies the game they will play def post(self): # calculate how likely they are to win # right now we only care about the goo

Routing application = webapp.WSGIApplication([ (‘/upload’, AddGames), (‘/stats’, GameStats), (‘/predict’, PredictGame)])

Thankfully Most requirements have very little state shared between URLs (all saved in DB) Thus each class is one of two forms: get / post: Grab data from user and do something with it get: Just display or synthesize some data

Rendering Web Pages Two main options available: self.response.out.write(“Hello World!”) self.response.out.write(template.render(‘index.html’, {‘my_title: ‘baz’, ‘data’: some_variable})) Use the second for any non-trivial app

Rendering Web Pages Now in your index.html: <title> {{ my_title }} </title> <div> {{ data }} </div>

Tips for Your Programs Separate out your code into files as needed app.yaml specifies this: handlers: - url: /one script: one.py - url: /two/(.*)/(.*) script: /two/\2/\1.py

Tips for Your Programs Make pages with sensitive data ‘secure’ All access done over HTTPS Optional - HTTP and HTTPS work Never - HTTPS redirects to HTTP Always - HTTP redirects to HTTPS

Tips for Your Programs Specify static files in app.yaml - url: /images static_dir: static/images Can also tag ‘secure’ if desired

Tips for Your Programs Be creative! The webapp framework supports: get, post, head, options, put, delete, trace Map user pages to ~name?

Summary Google App Engine is more than flexible enough to handle your web app desires And it’s only getting better! More than enough possibilities to create a web app: you just need a good idea!

Designing the Call of Cthulhu app with Google App Engine

More Related Content

What's hot (14)

Viewers also liked (15)

Similar to Designing the Call of Cthulhu app with Google App Engine (20)

Recently uploaded (20)

Designing the Call of Cthulhu app with Google App Engine