Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Google らしさ」
4 likes1,418 views
The document highlights the innovations and capabilities of Google Cloud Platform, particularly focusing on Google BigQuery and Google App Engine as tools for handling big data and application development. It emphasizes the performance, scalability, and ease of use offered by these services, along with insights into Google's container technology and global infrastructure. Notable statistics and testimonials underscore the efficiency and effectiveness of the platform in supporting large-scale applications and data processing.
Google Developers Summit Tokyo - Google Cloud Platform で知る Google クラウドの「Google らしさ」
- 1. Kazunori Sato Developer Advocate, Cloud Platform The Googliness of Google Cloud Platform
- 2. +Kazunori Sato @kazunori_279 Developer Advocate, Cloud Platform - Developer Community Support - gcp ja night, Docker Meetup, GCPUG - Product Launch Support
- 5. Cloud Technology Innovations 2012 2013 MapReduce Spanner/F1 2003 2006 2007 2010 2011 GFS Omega Colossus Cloud Storage Dremel BigQuery Bigtable Cloud Datastore Paxos impl. 2004
- 6. Management MobileDeveloper Tools Compute Networkin g Big Data Storage
- 7. Google Compute Engine Google App Engine Google BigQuery Data Analytics with Google’s massively parallel query engine IaaS with Google’s network and computing infra PaaS with Google’s container technology
- 9. 72 hours / min 100 petabytes 500M active users How Big is Big Data in Google?
- 10. In Google, we use Google BigQuery for processing Big Data SELECT top(appId, 20) AS app, count(*) AS count FROM installlog.2013 ORDER BY count DESC Scans 100B rows in 10 sec, without index.
- 11. Google BigQuery: Massively Parallel Query Service
- 12. Cost of BigQuery Storing 1TB = $26 / month Query on 10GB = $0.05 / query
- 13. select top(title), count(*) from publicdata:samples.wikipedia To scan 1 TB in 1 sec, you need 5,000 disks in parallel Massively Parallel Processing
- 14. Mixer 0 Mixer 1 Mixer 1 Shard Shard Shard Shard ColumnIO on Colossus SELECT state, year COUNT(*) GROUP BY state WHERE year >= 1980 and year < 1990 ORDER BY count_babies DESC LIMIT 10 COUNT(*) GROUP BY state Tree structure for fast aggregation
- 15. Streaming API + Fluentd: real-time import at 1M+ rows/s
- 17. Google App Engine: PaaS from Google
- 18. Google confidential | Do not distribute Google’s Container Technology Every Google service runs in containers Launches 2B containers every week
- 19. Google confidential | Do not distribute Scaling Out by Google’s Container Technology Instantiation in 40 ms
- 20. GAE Architecture And Optmization The Traditional Web Development Hardware Failures Traffic Spike Growing Big Data Complex Design Complex Dev. Complex Admin Cost
- 21. GAE Architecture And Optmization Google App Engine Encapsulates Them All Hardware Failures Traffic Spike Growing Big Data Simpler Design Simpler Dev No Admin No Initial Funding
- 22. Source: Google Internal Data 4.75 Million active applications 作用中應用程式 active applications
- 23. 700million “App Engine enabled us to focus on developing the application. We wouldn’t have gotten here without the ease of development that App Engine gave us.” Bobby Murphy, CTO Snapchat sends photos and videos each day Google App Engine scaled seamlessly during growth to millions of users Small team is able to innovate quickly and expand globally
- 25. The State of the Art Network Technology
- 26. Andromeda: The latest Software Defined Network (SDN) - Intra/Inter zone throughput = 9G bps - Inter regional private network by default
- 27. Google-scale Load Balancing region 2 region 3region 1 VMs VMs VMs :::::::::::::::::::::::::::::::::::::::::::::::HTTP LB DNS :) One global IP: 11.22.33.44 ● Load Balancing infra shared with Google.com ● Easily handles 1M req/s ● TCP anycast with single global IP ● Fast fail-over in seconds
- 29. Run and manage an optimized cluster Kubernetes Open Source Workload API & Tools Google Container Engine (GKE) Kubernetes ● Growing Community ● Linux containers support ● Open source orchestration create and manage Container Engine ● クラスタをすぐに起動可能 ● クラスタ環境の最適化 ● Kubernetesを実行管理する GKE Master GKE Nodes Compute Engine VMs Google Container Engine
- 30. Getting Started
- 32. Thanks !