TIAD : Automate everything with Google Cloud
4 likes3,406 views
The document provides an overview of Google Cloud Platform (GCP), highlighting its infrastructure, automation capabilities, and management options available for cloud resources. It discusses the use of containers, specifically Kubernetes, in Google's operations, as well as tools like Google Deployment Manager for configuring and managing resources. Additionally, it covers automation with third-party solutions such as Puppet, Chef, and Ansible for deployment and monitoring tasks within GCP.
TIAD : Automate everything with Google Cloud
- 1. Automate Everything with Google Cloud Platform Bastien Legras Cloud Platform Technical Lead bastien@google.com google.com/+bastienlegras @bastienlegras
- 2. Introduction to Google Cloud Platform
- 3. For the past 16 years, Google has been building out the world’s fastest, most powerful, highest quality cloud infrastructure on the planet.
- 4. Cloud Platform is built on the same infrastructure that powers Google.
- 5. Container Engine (Kubernetes) Today2002 2004 2006 2008 2010 2012 MapReduce Bigtable Dremel GFS Spanner Colossus Compute Engine App Engine
- 6. Google Cloud Platform NetworkingCompute Big Data Management Storage Mobile Developer Tools
- 7. ManagementNetworkingCompute Big Data Storage Mobile Developer Tools Google Cloud Platform Compute Compute Engine Container Engine App Engine
- 8. ManagementNetworkingCompute Big Data Storage Mobile Developer Tools Google Cloud Platform Networking Cloud Interconnect
- 9. ManagementNetworkingCompute Big Data Storage Mobile Developer Tools Google Cloud Platform Big Data Big Query Cloud Pub/Sub Cloud Dataflow
- 10. Google Cloud Platform Monitoring Management ManagementNetworkingCompute Big Data Storage Mobile Developer Tools
- 11. ManagementNetworkingCompute Big Data Storage Mobile Developer Tools Google Cloud Platform Storage Cloud Storage Cloud SQL Cloud Datastore
- 12. ManagementNetworkingCompute Big Data Storage Mobile Developer Tools Google Cloud Platform Mobile Firebase
- 13. ManagementNetworkingCompute Big Data Storage Mobile Developer Tools Google Cloud Platform Networking
- 15. Google Cloud Platform Management tasks ● Creating and destroying virtual instances ● Installing, configuring, and upgrading software on virtual instances ● Configuring disks, networks, and firewalls ● Configuring load balancing ● Monitoring running virtual instances ● Monitoring software on virtual instances
- 17. Google and Containers Everything at Google runs in a container. Internal usage: • Resource isolation and predictability • Quality of Services • batch vs. latency sensitive serving • Overcommitment • Resource Accounting We start over 2 billion containers per week. Image: "Container" glynlowe CC-BY-2.0 https://www.flickr.com/photos/glynlowe/10921733615
- 18. Let Me Contain That For You open source version of Google’s container stack github.com/google/lmctfy • Replacement for LinuX Container • Integrating with Docker (https://github.com/dotcloud/docker/pull/4891) • Separates policy from enforcement; buffers users from cgroups APIs • Programmable API and CLI
- 19. Declarative Over Imperative Imperative: "Start this container on that server" Declarative: "Run 100 copies of this container with a target of <= 2 tasks down at any time" Pros: • Repeatable • "Set it and forget it" • Eventually consistent • Easily updatable Con: • Tracing action/reaction can be difficult. "I made a change, is it done?" Image: "Space Needle under construction, 1961" seattlemunicipalarchives CC-BY-2.0 http://www.flickr.com/photos/seattlemunicipalarchives/6847114249
- 21. Google Cloud Platform Native Management Options UI CLI API Deployment Manager
- 22. Google Cloud Platform Google Deployment Manager GDM is an infrastructure management service that makes it simple to create, deploy, and manage Google Cloud Platform resources, using static or dynamic templates. https://cloud.google.com/deployment-manager/overview
- 23. Google Cloud Platform Google Deployment Manager Terminology ● Resource (ex: vm instance, sql instance) ● Configuration file (YAML) ● Templates : Python code or a Jinja template that defines a set of resources ● Manifest : read-only fully-expanded list describing all resources for a deployment. ● Deployment : collection of resources that are deployed and managed together, using a configuration file
- 24. Google Cloud Platform Google Deployment Manager Tutorial https://cloud.google.com/deployment-manager/create-advanced-deployment
- 25. Google Cloud Platform Template diskName: disk-created-by-cloud-config sourceImage: https://www.googleapis.com/compute/v1/projects/ debian-cloud/global/images/debian-7-wheezy- v20140619 networkInterfaces: - network: https://www.googleapis.com/compute/v1/projects/ myproject/global/networks/default - type: bigquery.v2.dataset properties: datasetReference: datasetId: example-id resources: - name: vm-created-by-cloud-config type: compute.v1.instance properties: zone: us-central1-a machineType: https://www.googleapis.com/compute/v1/projects/myprojec t/zones/us-central1-a/machineTypes/n1-standard-1 disks: - deviceName: boot type: PERSISTENT boot: true
- 26. Google Cloud Platform Tags & Startup Scripts Using instance tags $ gcloud compute instances add-tags INSTANCE --tags tag-1 tag-2 Using startup Scripts #! /bin/bash apt-get update apt-get install -y apache2 cat <<EOF > /var/www/index.html <html><body><h1>Hello World</h1> <p>This page was created from a simple startup script!</p> </body></html> EOF
- 27. Click to Deploy Software Packages
- 28. Google Cloud Platform 3rd Party Automation Partners https://puppetlabs.com/solutions/google-compute-engine https://www.chef.io/solutions/google-cloud-platform/ http://docs.saltstack.com/en/latest/topics/cloud/gce.html http://docs.ansible.com/guide_gce.html
- 29. Google Cloud Platform Google Compute Engine core runtime components
- 31. Google Cloud Platform Standalone Model
- 32. Google Cloud Platform Master/Agent Model Pull or Push model
- 33. Google Cloud Platform Master/Agent versus Standalone trade-offs Master/Agent allows : ● Single holistic view of your deployments ● Fine-grained access controls ● Group or role-based privilege separation ● Change logs for audit review ● Centralized views and reports ● Inventory categorization -> But it’s more complex harder to get started
- 34. Google Cloud Platform Deploy Puppet on Google Compute Engine with a click Posted: Wednesday, March 18, 2015
- 35. Google Cloud Platform The deployment of open-source Puppet ● Registering the Puppet repository with the package manager. ● Installing the Puppet master server and its dependencies. ● Configuring a firewall, if applicable. ● Starting the Puppet master and Puppet database. ● Setting service processes to start on boot. ● Making various changes required to properly operate Puppet, such as installing a patch to fix a bug in a module, removing a deprecated configuration, and avoiding red warnings.
- 37. Google confidential │ Do not distribute ● Single interface for monitoring all of your cloud resources ● Rich dashboards and alerting capabilities ● Find and fix performance problems quickly Cloud Monitoring
- 38. Google Cloud Platform Code deployment automation Connect to a github repo Automate code deployments
- 40. Google Cloud Platform References gcutil ● /compute/docs/gcutil/ Ansible ● http://www.ansible.com/ ● http://docs.ansible.com/gce_module.html Chef ● http://www.getchef.com/ ● https://github.com/opscode/knife-google Puppet ● https://forge.puppetlabs.com/puppetlabs/gce_compute ● http://puppetlabs.com/presentations/puppet-now-google ● http://puppetlabs.com/solutions/google-compute-engine ● http://docs.puppetlabs.com/pe/latest/cloudprovisioner_gce.html Salt ● http://www.saltstack.com/ ● http://docs.saltstack.com/en/latest/topics/cloud/gce.html
Editor's Notes
- #4: Data Center - For the past 14 years... » Where does Google fit in? It's simple. For the past 14 years we have been building out the world's fastest, most powerful, highest quality cloud infrastructure on the planet. Google powers Google - some of the largest, highest availability software products ever created and now we are bringing that to you.
- #5: Data Center - Cloud Platform is built on the same infrastructure that powers Google. » The Google Cloud Platform is us, productizing the same infrastructure, tools, processes, and systems that we used to build Google so that you can build your software and run your business in the same place. Enjoy the cost benefits. The quality. The reliability. And the performance. We're making it possible for you to use our infrastructure to power your business in an extremely scalable and efficient way.
- #6: Relentless focus on (re)defining what’s possible on the Web [ This slide builds ]: + How far we’ve come + Timeline | (select) Google infrastructure innovations - constantly pushing the boundaries for what’s possible + Spent billions of dollars making this the best web infrastructure in the world [ click for build ] + For those of you that are technical … examples of how we’ve externalized our innovations (eg. MapReduce) + GCP: AppEngine, Compute Engine + Containers + Latest announcements at GCPLive 2.0 (esp. containers)
- #18: Predictability =
- #19: This is a lower level component that most users won't interact with directly.
- #27: In this model, management of Compute Engine resources and instance software is performed from one or more workstations or laptops within your organization. In its most basic form, changes are explicitly initiated and pushed from a workstation. In a variant of the Standalone model, configuration information is pushed to a central repository, such as Google Cloud Storageor GitHub. Instances are set up with a simple scheduling tool such as cron to pull and apply the latest software and configuration.
- #30: In this model, management of Compute Engine resources and instance software is performed from one or more workstations or laptops within your organization. In its most basic form, changes are explicitly initiated and pushed from a workstation. In a variant of the Standalone model, configuration information is pushed to a central repository, such as Google Cloud Storageor GitHub. Instances are set up with a simple scheduling tool such as cron to pull and apply the latest software and configuration.
- #32: In this model, management of Compute Engine resources and instance software is performed from one or more workstations or laptops within your organization. In its most basic form, changes are explicitly initiated and pushed from a workstation. In a variant of the Standalone model, configuration information is pushed to a central repository, such as Google Cloud Storageor GitHub. Instances are set up with a simple scheduling tool such as cron to pull and apply the latest software and configuration.
- #33: In this model, administrators make configuration changes on the master, and the changes propagate to the managed instances to be applied locally. Managed instances report back to the master the success or failure of applying changes. Managed instances can receive software and configuration changes from the master in either a pull or push model. Pull model: The more traditional of the two approaches is the pull model. Agent software installed on the managed instance polls the master periodically for new configurations. Changes deployed in this model are expected to be applied "soon" (typically within a half hour). Push model: In this model, the master has the ability to push changes to agent nodes, generally using some form ofmiddleware such as a message queue. Changes deployed in this model are expected to be applied very quickly (within seconds or minutes).