GCCP - Session #3

Editor's Notes

• #4: [Read out the agenda on this slide.]
• #5: Welcome to So, What’s the Cloud Anyway?, I’m ________________.
• #6: Computers communicate with each other on a network. The computers in a single location, like an office, are connected on a local area network (LAN). Multiple locations can have their LANs connected to a wide area network, or WAN. Most networks today are connected to the internet, enabling millions of personal computers, servers, smartphones, and other devices to communicate, and provide and consume IT services.
• #7: Since around 2004, Google has been a leader in building out fast, powerful, high-quality cloud infrastructure. Google’s high-quality private network connects regional locations to more than 100 global network points of presence close to users. Google Cloud also uses state-of-the-art software-defined networking and distributed systems technologies to host and deliver services around the world. When every millisecond of latency counts, Google ensures that content is delivered with the highest throughput. https://cloud.google.com/about/locations/
• #8: Welcome to So, What’s the Cloud Anyway?, I’m ________________.
• #9: Virtual Private Cloud networks, or VPC, are used to build private networks on top of the larger Google network. With VPCs, you can apply many of the same security and access control rules as if you were building a physical network. VPCs allow the deployment of infrastructure as a service resources, such as compute instances and containers. They have no IP address ranges, are global, and span all available Google Cloud regions. VPCs contain sub-networks that span all zones in a region and can have default, auto, or custom modes. Sub-networks are also referred to as subnets.
• #10: Subnets are regional resources. They must be created in VPC networks to define sets of usable IP ranges for instances. VMs in different zones within the same region can share the same subnet. In this example, subnet1 is defined as 10.240.0.0/24 in the us-west1 region. Two VM instances in the us-west1-a zone are in this subnet. Their IP addresses both come from the available range of addresses in subnet1. Subnet2 is defined as 192.168.1.0/24 in the us-east1 region. Two VM instances in the us-east1-a zone are in this subnet. Their IP addresses both come from the available range of addresses in subnet2. Subnet3 is defined as 10.2.0.0/16, also in the us-east1 region. One VM instance in the us-east1-a zone and a second instance in the us-east1-b zone are in subnet3, each receiving an IP address from its available range. Because subnets are regional resources, instances can have their network interfaces associated with any subnet in the same region that contains their zones. A single VPN can be used to give private connectivity from a physical data center to the VPC.
• #11: Subnets are defined by an internal IP address prefix range and are specified in CIDR notation. CIDR stands for Classless Inter-Domain Routing. IP ranges cannot overlap between subnets. They can be expanded, but can never shrink. While IP ranges are specific to one region, they can cross zones within the region. You can also create multiple subnets in a single region. Although subnets don’t need to conform to a hierarchical IP schema, the internal IP ranges for a subnet must conform to RFC 1918.
• #12: Virtual machines that are in different regions but in the same VPC can communicate privately. VM1 and VM2 can communicate at a local level even though they’re separated geographically. Virtual machines that reside in different VPCs, even if the subnets are in the same region, need to communicate via the internet. In this instance, VM3 and VM4 will need public IP addresses to traverse the Internet. Networks don’t communicate with any other network by default.
• #13: Google Cloud offers two types of VPC networks, determined by their subnet creation mode. When an auto mode network is created, one subnet from each region is automatically created within it. As new Google Cloud regions become available, new subnets in those regions are automatically added to auto mode networks. The automatically created subnets use a set of predefined IP ranges and default firewall rules can be applied. In addition to the automatically created subnets, you can add more subnets manually to auto mode networks, in regions you choose, using IP ranges outside set of predefined IP ranges. When expanding the IP range in an auto mode network, the broadest prefix you can use is /16. Any prefix broader than /16 would conflict with the primary IP ranges of other automatically created subnets. Due to its limited flexibility, an auto mode network is better suited to isolated use cases, such as proof of concepts, testing, and so on. When a custom mode network is created, no subnets are automatically created. This type of network provides you with complete control over its subnets and IP ranges. You decide which subnets to create, in regions you choose, and using IP ranges you specify. You also define the firewall rules, and you can expand the IP range to any RFC 1918 size. Custom mode networks are therefore a lot more flexible and are better suited to production environments. While you can switch a network from auto mode to custom mode, this conversion is one-way. Custom mode networks cannot be changed to auto mode networks.
• #14: Each sub-network, or subnet, must be configured with a private IP CIDR address. CIDR stands for Classless Inter-Domain Routing. The CIDR range will determine what private IP addresses will be used by virtual machines in the subnet. Private IP addresses are only used for communication within the VPC and cannot be routed to the internet. Each octet in an IP address is represented by 8 binary bits. So a typical IPV4 address is 32 bits long. The number at the end of the range determines how many bits will be static or frozen. This number determines how many IP addresses are available with a CIDR address.
• #15: A /16 range will provide 65,536 available IP addresses. Everytime you add 1 to the last number, the number of available IP addresses is cut in half.
• #16: Internal IP addresses are allocated to VMs by a Dynamic Host Configuration Protocol (DHCP) service. The lease for the IPs are renewed every 24 hours. The name of the virtual machine is the host name, and the host name will be associated with the internal IP address through a network-scoped DNS service. External IP addresses can be ephemeral or reserved. They are assigned from a pool of IP addresses associated with the region. If you allocate a reserved IP address but don't attach it to a virtual machine, you will be billed for the IP address. Virtual machines are unaware of their public IP address. If you look at the operating system network configuration the virtual machine will only show the private IP address.
• #17: Virtual Private Cloud is a comprehensive set of networking capabilities and infrastructure that’s managed by Google. With Virtual Private Cloud, you can connect your Google Cloud resources in a virtual private cloud and isolate them from each other for purposes of security, compliance, and development versus test versus production. Cloud Load Balancing provides high performance, scalable load balancing for Google Cloud to ensure consistent performance for users. A content delivery network serves content to end users with high availability and high performance, usually by storing files close to the user. With Cloud CDN, Google’s global network provides low-latency, low-cost content delivery. Cloud Interconnect lets you connect your own infrastructure to Google’s network edge with enterprise-grade connections. Connections are offered by Google’s partner network-service providers, and may offer higher service levels than standard internet connections. Cloud DNS (Domain Name System) translates requests for domain names into IP addresses. Google provides the infrastructure to publish specific domain names in high-volume DNS services suitable for production applications.
• #18: Google Cloud firewall rules protect virtual machine instances from unapproved connections, both inbound and outbound, known as ingress and egress, respectively. Essentially, every VPC network functions as a distributed firewall. Although firewall rules are applied to the network as a whole, connections are allowed or denied at the instance level. You can think of the firewall as existing not only between your instances and other networks, but between individual instances within the same network. Google Cloud firewall rules are stateful. This means that if a connection is allowed between a source and a target or a target and a destination, all subsequent traffic in either direction will be allowed. In other words, firewall rules allow bidirectional communication once a session is established. Also, if for some reason, all firewall rules in a network are deleted, there is still an implied "Deny all" ingress rule and an implied "Allow all" egress rule for the network.
• #19: Conceptually, a firewall rule is composed of certain parameters. The direction of the rule. Inbound connections are matched against ingress rules only, and outbound connections are matched against egress rules only. For the ingress direction, sources can be specified as part of the rule with IP addresses, source tags, or a source service account. For the egress direction, destinations can be specified as part of the rule with one or more ranges of IP addresses. The protocol and port of the connection, where any rule can be restricted to apply to specific protocols only or specific combinations of protocols and ports only. The action of the rule, which is to allow or deny packets that match the direction, protocol, port, and source or destination of the rule. The priority of the rule, which governs the order in which rules are evaluated. The first matching rule is applied. The rule assignment. By default, all rules are assigned to all instances, but you can assign certain rules to certain instances only. Let’s look at some Google Cloud firewall use cases for both egress and ingress.
• #20: Egress firewall rules control outgoing connections originated inside your Google Cloud network. Egress allow rules allow outbound connections that match specific protocol, ports, and IP addresses. Egress deny rules prevent instances from initiating connections that match non-permitted port, protocol, and IP range combinations. For egress firewall rules, destinations to which a rule applies may be specified using IP CIDR ranges. Specifically, you can use destination ranges to protect from undesired connections initiated by a VM instance towards an external destination. For example, an external host. You can also use destination ranges to protect from undesired connections initiated by a VM instance towards specific Google Cloud CIDR ranges. For example, a VM in a specific subnet.
• #21: Ingress firewall rules protect against incoming connections to the instance from any source. Ingress allow rules allow specific protocol, ports, and IP addresses to connect in. The firewall prevents instances from receiving connections on non-permitted ports or protocols. Rules can be restricted to only affect particular sources. Source CIDR ranges can be used to protect an instance from undesired connections coming either from external networks or from Google Cloud IP ranges. This diagram illustrates a VM receiving a connection from an external address, and another VM receiving a connection from a VM in the same network. You can control ingress connections from a VM instance by constructing inbound connection conditions using source CIDR ranges, protocols, or ports.
• #22: Cloud VPN securely connects an on-premises network to a Google Cloud VPC network through an IPsec VPN tunnel. Traffic traveling between the two networks is encrypted by one VPN gateway, then decrypted by the other VPN gateway. This protects data as it travels over the public internet, and that’s why Cloud VPN is useful for low-volume data connections. As a managed service, Cloud VPN provides an SLA of 99.9% service availability and supports site-to-site VPN. It doesn’t support client-to-gateway scenarios though. In other words, Cloud VPN doesn't support use cases where client computers need to “dial in” to a VPN using client VPN software.Cloud VPN also supports both static routes and dynamic routes to manage traffic between VM instances and an existing infrastructure.
• #23: Cloud Interconnect provides two options for extending an on-premises network to a Google Cloud VPC network. Cloud Interconnect - Dedicated, also referred to as Dedicated Interconnect, and Cloud Interconnect - Partner, also referred to as Partner Interconnect. Choosing an interconnect type will depend on connection requirements, such as the connection location and capacity.
• #24: Let’s compare the interconnect options just considered. All of these options provide internal IP address access between resources in an on-premises network and a VPC network. The main differences are the connection capacity and the requirements for using a service. The IPsec VPN tunnels that Cloud VPN offers have a capacity of 1.5 to 3 Gbps per tunnel and require a VPN device on the on-premises network. The 1.5-Gbps capacity applies to traffic that traverses the public internet, and the 3-Gbps capacity applies to traffic that is traversing a direct peering link. Configuring multiple tunnels allows you to scale this capacity. Dedicated Interconnect provides high-bandwidth connections with a minimum of 10 Gbps. Traffic flows directly between networks, not through the public Internet. Setup requires routing equipment in a Google-supported colocation facility that supports the regions that you want to connect to. Connection capacity is delivered over one or more 10 Gbps or 100 Gbps Ethernet circuits, with a maximum of 8 x 10 Gbps circuits (80 Gbps), or 2 x 100 Gbps (200 Gbps) circuits for each Dedicated Interconnect connection. Partner Interconnect has a capacity of 50 Mbps to 10 Gbps per connection, and requirements depend on the service provider. The recommendation is to start with VPN tunnels and, depending on the proximity to a colocation facility and capacity requirements, to switch to Dedicated Interconnect or Partner Interconnect when there’s a need for enterprise-grade connections to Google Cloud.
• #25: You can use load balancing to take more advantage of an augmented infrastructure. You’ve already configured networking between different virtual machines, but how are you going to route traffic between multiple virtual machines? Load balancing is the first thing that comes into play. HTTP(S), SSL proxy, and TCP proxy load balancing are global services, whereas network and internal load balancing are regional.
• #26: IAP lets you establish a central authorization layer for applications over TLS, so you can use an application-level access control model instead of relying on network-level firewalls. Applications and resources protected by IAP can only be accessed through the proxy by users and groups with the correct Cloud IAM role. The proxy provides a layer of protection between the outside world and an internal service. When you grant a user access to an application or resource by IAP, they’re subject to the fine-grained access controls implemented by the product in use without requiring a VPN. IAP performs authentication and authorization checks when a user tries to access a IAP-secured resource. IAP secures authentication and authorization of external requests via TLS. IAP doesn't protect against activity inside your VM, such as if someone accesses the VM via SSH. IAP also doesn’t protect against activity within a project, such as VM-to-VM communication within your project over the local network.
• #27: Welcome to So, What’s the Cloud Anyway?, I’m ________________.
• #28: Google Cloud's operations suite provides powerful monitoring, logging, and diagnostics for apps on Google Cloud. It equips you with insight into the health, performance, and availability of cloud-powered apps, enabling you to find and fix issues faster. Google Cloud's operations suite gives you access to many different kinds of signals from your infrastructure platforms, virtual machines, containers, middleware, and application tier: logs, metrics, traces. It gives you insight into your application’s health, performance, and availability, so if issues occur you can fix them faster.
• #29: Let’s start by looking at Cloud Monitoring, a full-stack monitoring service that discovers and monitors cloud resources automatically. Flexible dashboards and rich visualization tools help you to identify emergent issues. Anomaly reporting, pattern detection, and exhaustion prediction provide insights into longer-term trends that may require attention. Monitoring provides a single integrated service for metrics, dashboards, uptime monitoring, and alerting. This means you spend less time maintaining disparate systems. Advanced alerting capabilities, including the rate of change, cluster aggregation, and multi-condition policies, help ensure you are notified when critical issues occur while reducing the likelihood of false positives. Integrated uptime monitoring and health checks ensure quick notification of failures. It’s possible to drill down from alerts to dashboards to logs and traces in order to identify the root cause of problems quickly.
• #30: Cloud Logging is a fully integrated solution that works seamlessly with Cloud Monitoring, Trace, Error Reporting, and Debugger. The integration allows users to navigate between incidents, charts, traces, errors, and logs. This helps users quickly find the root causes of issues in their system and applications. Logging is built to scale and works well at sub-second ingestion latency at terabytes per second. Logging is a fully managed solution that takes away the overhead of deploying or managing a cluster, thus allowing users to focus their energy on innovation and building a product. Logging provides a central place for all your logs. You can also configure Logging to export logs to other systems automatically. Logging allows you to analyze high-volume application and system logs in real time. Advanced log analysis can be achieved by combining the power of the operations suite with the data and analytics products of Google Cloud. For example, you can create powerful real-time metrics from the log data and analyze log data in real time in BigQuery.
• #31: Error Reporting allows you to identify and understand application errors through real-time exception monitoring and alerting. Error Reporting allows you to see your application’s top errors in a dashboard. Real production problems can be hidden in mountains of data. Error Reporting helps you see problems through the noise by constantly analyzing exceptions and intelligently aggregating them into meaningful groups tailored to your programming language and framework. Error Reporting is constantly watching your service and instantly alerts you when a new application error cannot be grouped with existing ones. Directly jump from a notification to the details of the new error. The exception stack trace parser is able to process Go, Java, .NET, Node.js, PHP, Python, and Ruby. You can also use Google’s client libraries and REST APIs to send errors with Cloud Logging.
• #32: Cloud Trace is a distributed tracing system that collects latency data from applications and displays it in the Cloud Console. Using Cloud Trace, you can inspect detailed latency information for a single request or view aggregate latency for your entire application. You can quickly find where bottlenecks are occurring and more quickly identify their root cause. Trace continuously gathers and analyzes data from applications to automatically identify changes to an application's performance. These latency distributions, available through the Analysis Reports feature, can be compared over time or versions, and Trace will automatically generate an alert if it detects a significant shift in an application's latency profile. The language-specific SDKs of Trace can analyze projects running on VMs. The Trace SDK is currently available for Java, Node.js, Ruby, and Go, and the Trace API can be used to submit and retrieve trace data from any source. A Zipkin collector is also available, which allows Zipkin tracers to submit data to Trace. Cloud Trace works out-of-the-box on many Google Cloud services like App Engine.
• #33: Cloud Debugger is a feature of Google Cloud that lets you inspect the state of a running application in real time, without stopping or slowing it down. Debugger can be used with production applications. With a few mouse clicks, you can take a snapshot of your running application state or inject a new logging statement. A snapshot captures the call stack and variables at a specific code location the first time any instance executes that code. The injected log point behaves as if it were part of the deployed code writing the log messages to the same log stream. Debugger is easier to use when source code is available. It knows how to display the correct version of the source code when a version control system is used, such as Cloud Source Repositories, GitHub, Bitbucket, or GitLab. Users can easily collaborate with other team members by sharing their debug session. Sharing a debug session is as easy as sending the Console URL. Debugger is integrated into existing developer workflows. Users can launch Debugger and take snapshots directly from Cloud Logging, Error Reporting, dashboards, integrated development environments, and the gcloud command-line interface.
• #34: Poorly performing code increases the latency and cost of applications and web services every day. Cloud Profiler continuously analyzes the performance of CPU or memory-intensive functions executed across an application. While it’s possible to measure code performance in development environments, the results generally don’t map well to what’s happening in production. Many production profiling techniques either slow down code execution or can only inspect a small subset of a codebase. Profiler uses statistical techniques and extremely low-impact instrumentation that runs across all production application instances to provide a complete picture of an application’s performance without slowing it down. Profiler allows developers to analyze applications running anywhere, including Google Cloud, other cloud platforms, or on-premises, with support for Java, Go, Node.js, and Python.