Anton Grishko "Multi-cloud with Google Anthos, Kubernetes and Istio. How to stop worry about infrastructure and stop losing money."

Editor's Notes

• #2: Hello all, We greatly appreciate to see all of you here today. Thanks all for joining us. First of all, we would like to introduce ourselves. Hi, I'm Ruslan Kusov, DevOps Architect from SoftServe’s Critical Services-Center of Excellence group. I have more than 9 years of experience in the IT industry as a System Administrator, Network Engineer, DevOps/SRE, and DevOps Architect. I created and optimized cloud architecture for global tourism, payment services, and mobile development companies. Also I'm a Certified Google Cloud Professional Architect. Hi, I'm Anton Grishko, DevOps architect from SoftServe’s Critical Services-Center of Excellence group. I have 12 years of experience working as a System Administrator, DevOps Architect, and Head of DevOps. My main specialization is migration projects with zero downtime. I've helped to establish a DevOps culture in multiple Fortune 100 companies as well as several Silicon Valley startups. And today we would like to share our own experience, SoftServe's best practices and show you how to do cost-effective A/B testing with Istio and Google Kubernetes Engine. 
• #3: We would like to dive deeper into this and choose following topics to talk today: 1)    We will start with GCP cloud and focus in detail on managed services themselves. We hope that at the end you'll clearly understand why it's so important to not ignore modern cloud services and how they can help your business and product. 2)    We will do intro to Kubernetes, it's components and design principles. We will show you that Kubernetes is really powerful tool that can bring your product a lot of advantages. Kubernetes is a good choice for microservices architecture, but at the same time it is quite complicated in installation, maintenance and support. And it's great that in Google Cloud we have Container as a service solution - Google Kubernetes Engine - or GKE. We will focus on GKE basic features and highlight all it's benefits for your workload. 3)    Ok, so we decided to proceed with Kubernetes and microservices architecture. But how can we improve this solution, make it more powerful, controllable, secure and easily adjust to microservices requirements. Istio can help us with this. We will help you to learn what is service mesh. We will talk about Istio, which is one of the most popular service mesh solutions. And at the end we will discuss A/B testing and demonstrate you how to do this in easiest and cost effective way with a power of Google Cloud, Google Kubernetes Engine and Istio. 4)  After this we are going to introduce Anthos and share our own experience about this "young" but at the same time really progressive and forward-looking technology. 4)    And at the and we would be happy to answer your questions, discuss your particular cases and help with advice for solutions. So let's get it started. 
• #4: And let's do this with the discussion about Google Cloud Platform and managed services. 
• #5: Having this significant background we understand how it can be difficult to build modern, really cost effective and production ready solution. It is challenging for startups and mature enterprises likewise. Either you start from scratch, has not enough background, but realize how important is to be flexible, easily react on continuously changed requirements from your customers and survive in endless competition with business rivals. Or you want to migrate your current on-premise data center to the Google cloud, make your applications cloud-native to decrease capital expenditures, reduce time-to-market for new features and services, and release in more predictable way.   
• #6: It's a struggle, because services with multiple component interactions and third-party dependencies require careful planning. Any additional demands, like specific business logic or microservice requirements create an extra challenge. Let's have a look at these requirements: -    traffic shaping​ -    Routing -    Advanced load balancing​ (including Blue-Green deployment and A/B testing purposes) -    Service discovery​ -    Circuit breaking​ -    rate limiting​ -    Metrics and Monitoring -    Logging -    Tracing​ Wow. It scares, isn't it? Hm-m-m, maybe. But I don't want to frighten all of you today. So don't worry, you'll feel good and pretty familiar at the end of our meetup. We've been asked about microservice architecture and cost effective solutions hundreds of times. For start-ups or enterprises, whatever. We still have the same answer, yes, it is possible to build it in any cases. If you are not versed about technologies, have not enough time or resources to fill knowledge gaps, or learn them and implement as a solution, you definitely should choose managed services. If you want to avoid maintenance "headache", you want to sleep better at night, don't want to be emergency awakened by support team, you have to choose managed services. 
• #7: This is a trade-off, which is implemented as a shared responsibility model between you and Google Cloud. But Ruslan, I heard a lot about flip side of managed services. Could you please explain me where is that border between vendor locking and "maintenance and support headache"? Yes, sure Anton, no problem. And as a great example, let's look closer at our today's topic and talk about Kubernetes. Kubernetes is Greek for "helmsman" or "captain" is a person who steers a ship. And the difference between management of self hosted Kubernetes cluster and GKE solution is like between steering frigate in middle ages and modern cruise ship nowadays respectively. I’ll show you this in details. Anton, so what do we know about Kubernetes? 
• #8: Kubernetes is open-source container management system originated and open-sourced by Google in 2014. It meets all the requirements to be a key tool to develop services in microservices architecture. Kubernetes has a large and rapidly growing community of users and contributors. It is advanced and mature orchestrator. Kubernetes services and tools are available, fully documented and well supported. It allows to create environments in seconds, using namespace isolation. It supports infrastructure as code ideas. Kubernetes runs on any cloud and bare metal. Yes, Kubernetes provides container-centric infrastructure. We are talking about control and management on containers level, not a VM. Main Kubernetes features are: -    Scheduling: It decides where my containers should run -    Lifecycle and health: Kubernetes keeps our containers running despite failures -    Scaling: It makes sets of containers bigger or smaller, depending on incoming load for example -    Naming and discovery: Kubernetes finds where our containers are now -    Load balancing: It distributes traffic across a set of containers -    Storage volumes: Kubernetes provides data for containers -    Logging and monitoring: It tracks what’s happening with our containers -    Debugging and introspection: Kubernetes allows to enter or attach to containers -    Least but not last, Identity and authorization: It controls who can access our containers 
• #9: Or in simple terms, Kubernetes provides features similar to an OS for a host: -    Scheduling workload -    Finding the right host to fit your workload -    Monitoring health of the workload -    Scaling it up and down as needed -    Moving it around as needed I hope, that everyone has already got Kubernetes “treasure map” and had a chance to examine it in detail. Now you see how powerful Kubernetes is. How many features it has. Just look at this: Configville, Applandia, Volume Beach, GKE ClusterCity, Services Archipelago and finally Istio harbor. And all these features and services are real treasures for our projects. Let’s pretend pirates for few minutes and board this ship. Anton, but I think that it is not a trivial task to support and maintain all of the Kubernetes components. 
• #10: Exactly. Here is a structure of Kubernetes cluster. It contains 2 main components: master (control plane) and nodes (or workers). Users can interact with Kubernetes cluster via CLI, API calls, UI. But let's look deeper on master and nodes components. And let's start with workers. Everything is pretty simple there. Here we have kubelet, kube-proxy, Docker container run-time and monitoring/logging components. Clients use Kube Control (kubectl) CLI to interact with the cluster. It's fairly simple and not a problem at all to manage Kubernetes nodes, isn’t it? 
• #11: But what about deep dive to kubernetes masters components? This list is much longer. Here we have -       API server (kube-apiserver) -       etcd -  reliable distributed key-value store -       Scheduler (kube-scheduler) -       Controllers: -       Kube controller (kube-controller-manager) -       Replication controller -       Endpoints controller -       Service account and token controllers -       Cloud controller (cloud-controller-manager) -       Add-ons Now, Anton, just imagine that you need to manage whole Kubernetes cluster. How are you going to do this? Are you ready to be responsible for HA architecture? Manage (by yourself, of course) at least 3 masters through 3 zones in Google cloud region, be responsible for etcd sync and backup? Gosh, I remember my own experience, when I started with Kubernetes. We had self managed cluster and our etcd failed because of unexpected termination of one instance and etcd restart on another one. We spent about 5 hours, woke up in the middle of the night by emergency call from support team. That was awful. And what about immutable infrastructure and consistent environments? Are you ready to create Terraform or Deployment manager configurations + Puppet manifests or Ansible playbooks to provision all that stuff? And are you ready to perform cluster updates? Especially for minor, not patch versions. I don’t think so. I believe, you have no reason for this. You respect your time and understand the border line of your responsibility. Ok, Ruslan. I understand what you mean. You try to highlight all pros of managed services. Exactly and as an example I would like to dive deeper in GKE – or Google Kubernetes Engine. 
• #12: What is it? In simple words it is Kubernetes Cluster managed by GCP. Formerly known as Google Container Engine, GKE is the easiest & fastest way to production Kubernetes. gcloud CLI (part of the Google Cloud SDK) can provision a functional Kubernetes cluster in minutes. It tracks OSS Kubernetes typically < 1 week after every minor release. And here let’s get back to our comparison. Remember, I told you that Kubernetes is Greek for "helmsman". For a person who steers a ship. And the difference between management of self hosted Kubernetes cluster and GKE solution is like between steering frigate in middle ages and modern cruise ship nowadays respectively. So yes. In middle ages to be a good captain you had to know everything about sea navigation, the laws of physics. They knew exclusive technique of the wood processing. Captains created their own maps and built their ship from scratch, update and repair them after cruises. The same with self hosted Kubernetes. You need to know how it’s operate in detail. You build from scratch your own cluster, configure etcd, maintain and update it, etc. On modern ship all what you need as a captain is to set destination port and after that control and monitor all ship systems and navigation. And here is our analogue from Kubernetes world – Google Kubernetes Engine or shortly GKE. 
• #13: So GKE architecture suppose that you work with fully managed Kubernetes control plane backed by SLA without necessity to operate and maintain masters. All automated operations tasks built-in, like node upgrades/node repair. Non-overlay networking is native to GCP And last but not least you have opinionated Kubernetes configuration. GKE architecture of cluster nodes is managed by node pools, a group of nodes with alike configuration: -    You can configure one or multiple node pools per cluster -    You can use node pools for regular VMs or preemptible VMs (another cool GCP feature that allows you to build cost effective solutions) -    You can configure multi-zone node pools, even with a single zone master -    And finally, it Is controlled by Managed Instance Groups, a construct in Google Compute Engine 
• #14: What about GKE Architecture of master nodes. Kubernetes Master backed by 99.5% SLO, It is single VM running all control plane components in a Google-managed project, all upgrades are automatic, you can choose to opt-in to upgrade earlier, all masters are scaled automatically to accommodate cluster size, masters are currently scoped to a single zone, high-availability clusters are in Beta to replicate the master across three zones in a single region, that will increase the SLO to 99.99%. 
• #15: Ok, let me draw a conclusion. What ships with a Kubernetes Engine Cluster? -       Firstly - Master Node (API Server, etcd Scheduler, Controller-Managers) -       Secondly - Nodes running container-optimized OS or Ubuntu -       Kube-dns deployment -       Kube-dns autoscaler deployment -       Event-exporter deployment logging to Stackdriver Logging -       fluentd daemon set logging to Stackdriver Logging -       Heapster deployment writing to Stackdriver Monitoring 
• #16: Ok, now it looks like we have all required information about the idea of managed services. So let’s get back to our question: Are we ready to production? Managed services are not always bad. There is a border between vendor locking and the "maintenance and support headache." You still have your Kubernetes under the hood, use the same templates for deployments, services, etc. Operate with the same docker images from your own docker registry, still use your helm charts. Think about GKE cluster features: Ease of Management / Reliability / Performance / Security / User Experience And think that now you can go to production faster and have shorter on-boarding period for your team members. Anton, are we good? Did Kubernetes and GKE solve all our problems and cover all the requirements to microservices? 
• #17: No Ruslan, not exactly. We still have a half of this list that cannot be handled by Kubernetes itself. And Istio will help us here. 
• #18: Meet Istio!
• #19: Maintaining resilience, discovery, and routing logic in code for independent services written in different languages becomes incredibly complex and expensive to operate The role of a service mesh is to overlay your services with a management framework. 
• #20: A service mesh differs from an edge/API service. A service mesh is an infrastructure built for service-to-service communication and resiliency with zero business logic. Here is the list of key features of service mesh: ·      Routing/traffic shaping ·      Advanced load balancing ·      Service discovery ·      Circuit breaking ·      Timeouts/retries ·      Rate limiting ·      Metrics/logging/tracing ·      Fault injection Oh, now I see how they aligns with microservices requirements and supplement Kubernetes. Yep, exactly. Let’s look at service mesh conceptual architecture. 
• #21: A service mesh architecture is comprised of two parts: Control plane — configures the service proxies and manages the mesh Data plane — acts as a service proxy and communicates service behaviour back to the control plane. Anton, I would like to ask our audience. Do you know any service mesh solution? Please, raise your hands. Ok, thanks. Have you already tested some of them? Do use any service mesh solution on your production environments? Could you please tell us what solutions do you use? Ok. So we would like to provide you with new knowledge and information about, probably, one of the famous and mature service mesh solution nowadays – Istio. 
• #22: Istio is an open-source service mesh from Google and IBM that provides service discovery, security, instrumentation, intelligent client-side load balancing, circuit breakers, and dynamic routing with a pluggable architecture for telemetry and policy. Istio uses the Envoy proxy to transparently add in the service mesh functionality. Istio provides uniform observability, mTLS-based security, and traffic management. Istio can be used with both Kubernetes-based workloads as well as workloads deployed on VMs. And moreover. Istio is now available as a service with Google Kubernetes Engine. 
• #23: Anton, I propose us to go through Istio key operation topics and tell us in details next: -    Securing using RBAC Istio role-based access control (RBAC) provides access control for services in Istio Mesh -    Mutual TLS authentication Istio's mutual TLS authentication architecture provides a strong service identity and secure communication channels between services -    Monitoring Istio Monitoring tools (e.g., Prometheus, Stackdriver) provide monitoring for both Istio control plane components and transparent proxies -    And last but not least A/B testing Istio provides with a variety of traffic management and dynamic request routing for A/B testing, gradual rollouts, and canary releases, it also handles failure recovery using timeouts, retries, circuit breakers, and fault injection 
• #24: Sure Ruslan. Let’s do this. And start with Istio Role Based Access Control. Pay your attention to this diagram. Istio role-based access control (RBAC) provides namespace-level, service-level, and method-level access control for services in Istio Mesh. It features: ·      Role-based semantics, which is simple and easy to use ·      Service-to-service and end user-to-service authorization ·      Flexibility through custom properties support in roles and role-bindings 
• #25: Next feature is Istio mutual TLS or mTLS Authentication. Istio Authentication’s aim is to enhance the security of microservices and their communication without requiring service code changes. It is responsible for: ·    Providing each service with a strong identity that represents its role, enabling interoperability across clusters and clouds ·    Securing service-to-service communication and end user-to-service communication ·    Providing a key management system to automate key and certificate generation, distribution, rotation, and revocation I remember one of my project, where we had a requirements to mTLS and secure communication between services, but we didn’t use Istio there. It was real pain in neck. We had to generate, revoke, update and properly store certificates and keys, make a lot of code changes to support these certificates and mTLS for application. If I’d known Istio deeply and had more experience with it, I would have used Istio service mesh on that project. I wish I could turn the clock back. Ok Ruslan, don’t worry. You did everything you could. Next time will be better. 
• #26: Let’s focus now on the other Istio feature and talk about Istio monitoring. What do we have here: ·      Mixer — responsible for telemetry reporting ·      Prometheus — querying Istio metrics provided by Mixer ·      Grafana — visualizing metrics in a graphical dashboard ·      Zipkin — to collect traces and then send to Stackdriver
• #27: And finally let’s talk about A/B testing with Istio. A/B testing deployments route a subset of users to a new functionality under specific conditions. It is usually used for business analysis, to make decisions about product changes. Possible list of conditions to distribute traffic amongst the app versions include: ·    Weight ·    Cookies ·    Query parameters ·    Geolocation ·    User-agent (browser version), operating system, screen size, user device, etc. ·    User language 
• #28: Now, Anton, I have a question to you. But this time it is more global. And I have it because I’ve heard a lot of times from our customers following questions: “Can we design and build hybrid and multi-cloud architecture with managed services? Is it possible to launch GKE and Istio for on-premises environment or integrate with Kubernetes/Istio solutions in other clouds?” So my question is: Are we ready to this future? Multi-cloud and hybrid environments? How Google can assist us with this?
• #29: Yep, Ruslan. And I have an answer. Meet Anthos. Is this a new GCP technology that has been recently introduced? Yes, it was introduced on Google Next 19 event. 
• #30: Anthos is a modern application management platform that provides a consistent development and operations experience for cloud and on-prem environments. This page provides an overview of each layer of the Anthos infrastructure and shows how you can leverage its features. Anthos (formerly Cloud Services Platform) lets you build and manage modern hybrid applications across environments. Powered by Kubernetes and other industry-leading open-source technologies from Google, Anthos transforms your architectural approach, lets you focus on innovation, and allows you to move faster than ever without compromising security or increasing complexity. Become a strategic enabler of your business by modernizing your new and existing applications with containers, microservices architecture, and a service mesh delivered, and managed, by Google across your data center and the cloud. 
• #31: Modernize in place Migrate and modernize Automate policy and security at scale Consistent experience
• #32: Anthos Config Management has the following benefits for your Kubernetes Engine clusters: Single source of truth, control, and management Enables the use of code reviews, validation, and rollback workflows. Avoids shadows ops, where Kubernetes clusters drift out of sync due to manual changes. Enables the use of CI/CD pipelines for automated testing and rollout. One-step deployment across all clusters Anthos Config Management turns a single Git commit into multiple kubectl commands across all clusters. Rollback by simply reverting the change in Git. The reversion is then automatically deployed at scale. Rich inheritance model for applying changes Using Namespaces, you can create configuration for all clusters, some clusters, some Namespaces, or even custom resources. Using Namespace inheritance, you can create a layered Namespace model that allows for configuration inheritance across the repo folder structure.
• #33: Anthos is composed of multiple products and features. Below is a list of each component and its availability: Google Kubernetes Engine (GA) GKE On-Prem (Beta) Google Cloud Platform Marketplace (Beta) Anthos Config Management (Beta) GKE Hub (Alpha) Cloud Service Mesh (Alpha)
• #34: Anthos is composed of multiple products and features. Below is a list of each component and its availability: Google Kubernetes Engine (GA) GKE On-Prem (Beta) Google Cloud Platform Marketplace (Beta) Anthos Config Management (Beta) GKE Hub (Alpha) Cloud Service Mesh (Alpha) OUR EXPERIENCE 
• #35: Let me summarize what we have right now with Anthos: Single centralized place to control all environments (cloud and on-premises) Single centralized place to control traffic across all environments (cloud and on-premises) Single centralized place where configuration files are stored - control application configuration across all environments (cloud and on-premises) 
• #36: Ok, now it’s time for your questions. 
• #37: Thank you for joining us today. We really appreciate your time and we hope that you got new information and food for thought after today’s meetup. Don’t hesitate to ask us if you have any further questions. Share with us your own experience, your problems and pain and we would try to help you and assist with choosing of proper solutions for your particular cases. Thank you and see you next time.