Google App Engine - exploiting limitations
1 like574 views
Google App Engine is a PaaS that allows developers to build and host web applications on Google's infrastructure. It provides automatic scaling and simulates a production environment through its SDK. However, applications must comply with App Engine's rules, including a read-only filesystem, 30 second response times, and no socket connections or processes. Data is stored in Google's distributed Datastore, which uses entities instead of tables and does not support joins, aggregation, or transactions across entity groups. To work within these limitations, developers must change their approach to focus on the Datastore's strengths like simplicity and denormalize their data models.
![Python API
class Person(db.Model):
first_name = db.StringProperty()
last_name = db.StringProperty()
hobbies = db.StringListProperty()
p = Person(first_name = "Albert", last_name = "Johnson")
p.hobbies = ["chess", "travel"]
query = db.Query(Person)
query.filter("last_name = ", "Johnson")
results = query.fetch](https://image.slidesharecdn.com/appengineen-091015044243-phpapp01/85/Google-App-Engine-exploiting-limitations-10-320.jpg)
Google App Engine - exploiting limitations
- 1. Google App Engine exploiting limitations Tomáš Holas
- 2. What is App Engine? • Run your web applications on Google’s infrastructure • PaaS = Platform as a Service • Automatic scaling • Need to comply to platform’s rules • Application is sandboxed • Production environment simulated by SDK • Python, Java, and other JVM languages • Supports most web frameworks
- 3. What are the rules? • Read-only filesystem • 30s time to respond • No socket connections • No processes or threads • No system calls • Limit on number and size of files • Quotas on system resources
- 4. Everything through API • URL Fetch • Mail • Images • Google Accounts • CRON • Task Queues • XMPP (Jabber) • Memcached
- 5. Google Datastore • Distributed database • Base on BigTable • Supports transactions and partitioning • Supports hierarchies • Not a relational database • No schema • No SQL
- 6. Datastore principle • Primary building block: Entity • All entities stored in one “table” • Each entity has it’s Kind • instead of being in a separate table • Entity’s key is unique across Kind’s • Entity Groups - hierarchical structures • Entity has 0..N Properties
- 7. Vlastnosti Entit • Each Entity can have any combination of attributes. • Two entites of the same kind can have different properties • Multivalue properties are supported (ie. Array) • this can be used for 1:N and M:N relations • Each entity can have it’s parrent • It’s possible to construct hierarchical structures • Hierarchies are used for automatic partitioning of the database • Transactions are possible only across Entity Group members
- 8. Datastore limitations • Not using SQL, only GQL (Python) or JDOQL (Java) • No joins support, not even anything similar • No database constraints • No aggregation functions (count, avg, min, max…) • Maximum of 1000 records from one query. It’s not possible to go over this limitation using offset, but cursors support is planned. • Inequality filter on one property only SELECT * FROM Person WHERE birth_year >= 1980 AND birth_year <= 2009 SELECT * FROM Person WHERE birth_year >= 1980 AND height >= 180 • No global transactions - only inside Entity Group • No ad-hoc query support. Each type of query needs to have it’s own index configured. SDK does this automatically.
- 9. How to cope with this? • Consider suitability of the platform • Is it even possible to implement my project? • Change mindset • Change the way application is designed • Forget relational database stereotypes • Exploit Datastore’ • Simplicity, speed, versatility, multivalue attributes… • Denormalize! • Transfer the data consistence responsibility from database to application
- 10. Python API class Person(db.Model): first_name = db.StringProperty() last_name = db.StringProperty() hobbies = db.StringListProperty() p = Person(first_name = "Albert", last_name = "Johnson") p.hobbies = ["chess", "travel"] query = db.Query(Person) query.filter("last_name = ", "Johnson") results = query.fetch
- 11. Java API @PersistenceCapable(identityType = IdentityType.APPLICATION) public class Person { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Long id; @Persistent private String firstName; @Persistent private String lastName; @Persistent List<String> hobbies; } Query query = pm.newQuery(Person.class); query.setFilter("lastName == 'Johnson'"); List<Employee> results = (List<Employee>) query.execute();
- 12. DEMO !
- 13. Conclusion • App Engine easy development of scalable web applications • Be careful of specific limitations • Suitable for very small or very large projects • It’s getting better every day
- 14. Thank you! tomas.holas@gmail.com linkedin.com/in/tomash facebook.com/tomas.holas twitter.com/tomaash