Bogdan botea, dmitry nefedkin no fiddle, efficient development on the google cloud
1 like543 views
The document provides an overview of Appsbroker, a UK-based Google reseller, consultancy, and managed services provider. It discusses Appsbroker's work with Google on prestigious projects and certification as a Google Premier Partner. The rest of the document focuses on how efficient development can be achieved on the Google Cloud Platform, including how Appsbroker's Appsflow application was redesigned to take advantage of services like App Engine, Cloud Endpoints, Cloud Datastore, Task Queues, and Memcache. Live coding and a Q&A session are part of the agenda.
![Images by Connie Zhou
Wired, 'Google Throws Open Doors To Its Top Secret Data Center', October 2012
Google's Platform
"[Google's] ability to build, organize, and
operate a huge network of servers and
fiber-optic cables with an efficiency and
speed that rocks physics on its heels.
This is what makes Google Google: its
physical network, its thousands of fiber
miles, and those many thousands of servers
that, in aggregate, add up to the mother of
all clouds."
- Wired](https://image.slidesharecdn.com/bogdanboteadmitrynefedkin-nofiddleefficientdevelopmentonthegooglecloud-141120132110-conversion-gate01/85/Bogdan-botea-dmitry-nefedkin-no-fiddle-efficient-development-on-the-google-cloud-14-320.jpg)
Bogdan botea, dmitry nefedkin no fiddle, efficient development on the google cloud
- 1. No fiddle, efficient development on the Google Cloud Platform Dmitry Nefedkin Google Cloud Platform Solutions Engineer, Google Bogdan Botea Lead Developer, Appsbroker Consulting
- 2. ● Founded 7 Years ago, UK based ● Google Reseller, Consultancy, Managed Services & Products ● Google EMEA Partner of the year 2012 ● Worked on some of Google’s most prestigious projects ● UK Premier Apps Reseller, Cloud Platform Service Provider, We are also a Premier Search Reseller ● GEO, GSA, Platform & Apps Certified Who are Appsbroker?
- 3. Please note that some of the slides have been removed. Please come to our next event to get updated on the latest information about Google for Work and Appsbroker.
- 4. Agenda ■ Google awesome infrastructure - Dmitry ■ Before the clouds ■ Appsflow on App Engine ■ Live coding ■ Q&A
- 6. Google confidential | Do not distribute Enrich your work environment. Enable your information workers. Drive innovation. Live in a modern workplace. Why Cloud Computing?
- 7. Google confidential | Do not distribute Enterprise Cloud Platform market will exceed $22B globally by 2015. 2013
- 8. IT Trends The decreasing cost of storage enables virtually limitless storage in the cloud. $600 can buy enough storage for the world’s music. (Source: McKinsey Global Institute May 2011) Computing as a utility is now available for easy purchase, provided from massively efficient data centers. (Source: Nicholas Carr, The Big Switch, 2008) The internet allows for a model of real-time access to new innovation, information and applications from a wide range of devices. Affordable Capacity On-demand computing Instant access
- 9. For the past 16 years, Google has been building out the world’s fastest, most powerful, highest quality cloud infrastructure on the planet.
- 10. Cloud Platform is built on the same infrastructure that powers Google.
- 11. Google's Network Spans the Globe
- 12. Google's Global OpenFlow Network
- 13. 2002 2004 2006 2008 2010 2012 Google Innovations in Software ColossusDremelMapReduce SpannerGFS Big Table
- 14. Images by Connie Zhou Wired, 'Google Throws Open Doors To Its Top Secret Data Center', October 2012 Google's Platform "[Google's] ability to build, organize, and operate a huge network of servers and fiber-optic cables with an efficiency and speed that rocks physics on its heels. This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds." - Wired
- 15. IaaS PaaS SaaS Infrastructure-as-a-Service Platform-as-a-Service Software-as-a-Service Google Cloud Platform Cloud Computing
- 16. IaaS PaaS SaaS Infrastructure-as-a-Service Platform-as-a-Service Software-as-a-Service Google Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Packaged Software Applications Data Runtime Middleware O/S Virtualization Servers Storage Networking Cloud Computing You Manage Vendor Managed
- 17. Storage App ServicesCompute Cloud Storage Cloud SQL Cloud Datastore Compute Engine App Engine BigQuery Cloud Endpoints Google Cloud Platform
- 18. Mobile Gaming Big Data Storage High Performance Computing Digital Marketing
- 19. 4 Million active applications in our cloud
- 20. And your application can be the next! Dmitry Nefedkin Google Cloud Solutions Engineer nda@google.com
- 21. No fiddle, efficient development on the Google Cloud Platform Bogdan Botea Lead Developer, Appsbroker Consulting Vasile Irimia Google Cloud Platform Developer, Appsbroker Consulting
- 22. Appsbroker team
- 23. Agenda ■ Google awesome infrastructure - Dmitry ■ Before the clouds ■ Appsflow on App Engine ■ Live coding ■ Q&A
- 25. Flip the focus
- 26. Copyright 2014 Google Inc GAE Architecture And Optmization Use Case - Real Time Earthquake Monitor NIED App Engine Metrics collected at NIED NIED pushes the image file to GAE every second Clients pulls the image every two seconds ● GAE Web App By NIED Japan + Google o National research Institute for Earth science and Disaster prev. o The blinking dots represents real time Peak Ground Acceleration ● YouTube video How it worked at the March 11, 2011 earthquake ● 20,000 concurrent users 10,000 reqs/sec at peak
- 27. Web Application challenges ● Scalability ● Reliability ● Cost efficiency Designing for Scale and Reliability How do I design a web service that is scalable and reliable like this?
- 28. Copyright 2014 Google Inc GAE Architecture And Optmization To Build in Traditional Way... ➢ Hardware Failures ➢ Traffic Spike ➢ Growing Big Data ➢ Complex Design ➢ Complex Dev. ➢ Complex Admin ➢ Cost What About
- 29. Copyright 2014 Google Inc GAE Architecture And Optmization To Build with Google App Engine ✓ Hardware Failures ✓ Traffic Spike ✓ Growing Big Data ✓ Simpler Design ✓ Simpler Dev ✓ No Admin ✓ No Initial Funding How About
- 30. Agenda (to redesign) ■ Google awesome infrastructure - Dmitry ■ Before the clouds ■ Appsflow on App Engine ■ Live code ■ Q&A
- 31. Appsflow
- 32. Appsflow - Data model ● HRD Datastore ● Sharding ● Denormalization
- 33. Appsflow - HRD Datastore Datastore RDBMS Query language flexibility SQL-like query language ● Limited to simple filter and sort Full support of SQL ● Table JOIN ● Flexible filtering ● Subquery Reliability and Scalability Highly scalable and reliable Hard to scale
- 34. Appsflow - Service layer ● Business logic & transaction management ● DI ● Task Queue API ● Cron jobs
- 35. Appsflow - Task Queue Use Task Queue API to execute background work Queue queue = QueueFactory.getQueue("workflow-queue"); queue.add(TaskOptions.Builder.withUrl("/backend/taskqueue"). param("nodeId", node.getKeyAsString()));
- 36. Appsflow - Cron jobs Use Cron jobs to execute periodic tasks <?xml version="1.0" encoding="UTF-8"?> <cronentries> <cron> <url>/checkworkflows</url> <description>Periodically check for blocked workflows</description> <schedule>every day 00:00</schedule> </cron> </cronentries>
- 37. Appsflow - Cloud Endpoints ● Expose standards based REST interfaces with built in Authorization ● Use auto-generated, strongly typed, mobile optimized client libraries for Android, iOS and web.
- 38. Appsflow - Memcache MemchacheService cache = MemcacheServiceFactory.getMemcacheService(); … User user = cache.get(“accessToken”) ; if (user != null) { return user; } else { // retrieve user details and update the datastore cache.put(“accessToken”, user, Expiration.byDeltaSeconds(DURATION_IN_SECONDS)); return user; }
- 39. Snapchat Delivers 400 million photos a day
- 40. Agenda ■ Google awesome infrastructure - Dmitry ■ Before the clouds ■ Appsflow on App Engine ■ Live coding ■ Q&A
- 41. $500 in Cloud Platform credit to launch your idea! Build. Store. Analyze. On the same infrastructure that powers Google A voucher will be sent to your email Click ‘Apply Now’ and complete the application with promo code: codecamp-con 1 2 3 Go to cloud.google.com/starterpack Start using Cloud Platform now! 4 Create a new Project in Cloud Console, activate the billing and apply voucher - http://console.developers.google. com/billing/redeem