6 Things You Need to Know to Safely Run Kubernetes
2 likes549 views
This document is a presentation on safely running Kubernetes clusters. It discusses the need for soft multi-tenancy between namespaces, using a cloud-native approach to managing Kubernetes clusters, establishing clear roles and personas, handling stateful workloads, differences between Kubernetes implementations, and adopting new operational practices. The presentation provides examples and recommendations on each of these topics to help users run Kubernetes clusters safely and effectively.
6 Things You Need to Know to Safely Run Kubernetes
- 1. © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0 6 Things You Need to Know to Safely Run Kubernetes Cornelia Davis Vice President, Technology, Pivotal April 2019
- 2. Cover w/ Image Me? Developer (wasn’t Ops) Web architectures for >10 years Cloud-native for 6+ years Cloud Foundry for 6+ years Discount code 40% off!: 40cloudnat https://www.manning.com/books/cloud-native-patterns @cdavisafc
- 5. Let’s have a look at some things you need to get right…
- 6. #1 - Soft Multi-tenancy
- 7. K8s Cluster Kubernetes Master
- 8. Kubernetes Master API Server Controller Manager DNS Scheduler … …
- 9. Kubernetes Master API Server Controller Manager DNS Scheduler … Kubelet Kube Proxy Kubelet Kube Proxy
- 10. Kubernetes Master API Server Controller Manager DNS Scheduler … Namespace 1: Kubelet Kube Proxy Kubelet Kube Proxy Namespace 2: These are all shared components That are not namespace aware!!!
- 11. Kubernetes Master API Server Controller Manager DNS Scheduler … Namespace 1: Kubelet Kube Proxy Kubelet Kube Proxy Namespace 2: Limitations: ● Noisy neighbors (workloads can affect other tenants) ● Tenants share the same network ● Tenants share DNS ● Tenants share Configuration ● …
- 13. Kubernetes Master API Server Controller Manager DNS Scheduler … Namespace 1: Kubelet Kube Proxy Kubelet Kube Proxy Namespace 2: The suggestion: Give each tenant their own set of K8s controller components.
- 14. Kubernetes Master API Server Controller Manager DNS Scheduler Kubelet Kube Proxy Kubelet Kube Proxy Tenant 1: Kubernetes Master API Server Controller Manager DNS Scheduler Kubelet Kube Proxy Kubelet Kube Proxy Tenant 2: Leverage 20 years of maturity in hypervisor- based security!
- 16. #2 - Cloud-native Cluster Management
- 17. Kubernetes Takes Care of Your Workloads Image Cache etcd K8s Master Worker Worker Worker Kubernetes Scheduler Desired State Actual State Replica Set Controller LoadBalancer
- 18. But who is taking care of your Kubernetes?
- 19. VMs are Monitored IaaSBOSH Worker AGENT Master AGENT etcd AGENT Message Bus Health Monitor Responses: pager email monitoring ressurector … BOSH Director Desired State Actual State Message Bus
- 20. VMs are Monitored IaaSBOSH Worker AGENT Master AGENT etcd AGENT Message Bus Health Monitor Responses: pager email monitoring ressurector … BOSH Director Desired State Actual State Worker AGENT etcd AGENT BOSH Director Message Bus
- 21. VMs are Monitored IaaSBOSH Worker AGENT Master AGENT etcd AGENT Message Bus Health Monitor Responses: pager email monitoring ressurector … BOSH Director Desired State Actual State CPI BOSH Director Message Bus Worker AGENT etcd AGENT
- 22. PKS does for your Kubernetes what Kubernetes does for your apps
- 23. #3 - Who are Your Personas?
- 24. Teams Delivering Outcomes Platform Team Application Team Iteratively building and delivering digital offerings to the consumer Enabling the app teams all while maintaining Security Compliance Resilience Cost Efficiency Your Application Code Virtualized Infrastructure
- 26. Warning!!! kubectl delete deploy kubectl delete node This is workload related. This is cluster related!
- 27. #4 - Stateful Services?
- 28. https://twitter.com/kelseyhightower/status/963413508300812295 But a even more has happened since then!
- 29. Kubernetes Master Stateless workloads: ● Can be moved around ● Aided by service discovery ● Don’t have a particular start order ● Bind to backing services for state
- 31. Kubernetes Master Stateful workloads: ● Start order dependencies ● Storage % & % &
- 32. Kubernetes Master Stateful workloads: ● Start order dependencies ● Storage Kubernetes abstractions: ● Stateful sets ● Persistent volumes/persistent volume claims % & % &
- 33. Kubernetes Master Stateful workloads: ● Start order dependencies ● Storage Kubernetes abstractions: ● Stateful sets ● Persistent volumes/persistent volume claims % & % &
- 34. Warning!!! Availability Zone 1 Availability Zone2 ? ? Storage classes have different behaviors Data replication needs to be handled out of band
- 35. #5 - K8s is K8s is K8s - right? (Spoiler alert: Nope)
- 37. You are probably doing multi-cloud kubectl PKS AKS GKE EKS
- 38. Kubernetes Master API Server Controller Manager DNS Scheduler … … There are 150 flags you can set on startup There are many different controllers that affect workload behaviors Your chosen machine types can affect your workloads (i.e. standard CPU vs. GPU) Cluster addons affect your workloads
- 39. You are probably doing multi-cloud kubectl PKS AKS GKE EKS Different cluster configurations -> Different workload behaviors
- 41. You are probably doing multi-cloud kubectl PKS PKS PKS PKS Same cluster configurations across all clouds
- 42. #6 - New Operational Practices
- 43. You are probably doing multi-cloud kubectl PKS AKS GKE EKS pks az gcloud aws 4 times the: - Skills - Scripts - Runbooks - …
- 44. Where should the abstractions lie? kubectl PKS PKS PKS PKS pks pks pks pks
- 46. Embedded OS (Windows & Linux) NSX-T CPI (15 methods) v1 v2 v3 ... CVEs Product Updates vSphere Azure & Azure StackGoogle CloudAWSOpenstack Pivotal Network “3Rs” Concourse Repair — CVEs Repave Rotate — Credhub Pivotal Application Service (PAS) Pivotal Container Service (PKS) Pivotal Services Marketplace Pivotal Function Service (PFS) Do you trust your software delivery supply chain? https://www.youtube.com/watch?v=1qcTu2QUtrU
- 48. Slaying Dragons
- 49. Slaying Dragons 1. Think about your tenancy needs 2.You need something to take care of your Kubernetes clusters 3.Great care needed in establishing roles and permissions 4.Stateful workloads work - need to be deliberate about persistence 5.There is no such thing as “Vanilla Kubernetes” 6.Establish new operational practices
- 50. Transforming How The World Builds Software © Copyright 2017 Pivotal Software, Inc. All rights Reserved.
- 51. Cover w/ Image Me? Developer (wasn’t Ops) Web architectures for >10 years Cloud-native for 6+ years Cloud Foundry for 6+ years Discount code 40% off!: 40cloudnat https://www.manning.com/books/cloud-native-patterns @cdavisafc