Kubernetes in Adform
Download as PPTX, PDF1 like61 views
The document discusses Adform's use of Kubernetes for container orchestration. It describes Adform's Kubernetes clusters including sandbox, development, staging, and production clusters. It discusses why Adform uses containers and Kubernetes for benefits like build locally-run anywhere, immutable infrastructure, and improved workflow. The presentation also covers technologies around Kubernetes like monitoring and provides a live demo of Adform's self-service portal and deployment workflow.
Kubernetes in Adform
- 1. Kubernetes in Adform Edgaras Apšega Technical Product Manager @apsega 1
- 2. Agenda • About Adform • Kubernetes clusters and architecture • Monitoring • Deployment workflow • Self-service for developers • Live Demo (hopefully someone made some donations to the demo gods) 2
- 3. 3
- 4. Why Containers and Kubernetes? 4 Build locally – run anywhere • Dependencies packaged inside a container image • Consistent software version and assets • Declarative Configuration Immutable Infrastructure • No user modifications • Quick rollbacks • Self-healing systems Workflow • Efficiency • Scalability • No VM maintenance (OS, security updates, etc.)
- 5. Kubernetes Clusters 5 Sandbox • Fully virtualized, running on OpenStack • Test object for new features • 3 VMs for etcd • 3 VMs for control plane • 3 VMs as nodes Development • 2 clusters • 10 physical machines • 3 VMs for etcd • 3 VMs for control plane • 1000+ containers • 25 CPU cores used Staging • 6 physical machines • 3 VMs for etcd • 3 VMs for control plane • 400+ containers Production • Multi AZ • 17 physical machines • 550+ cores • 5 VMs for etcd • 3 VMs for control plane • + 3 planned remote clusters • 700+ containers
- 7. Exposing service on Bare Metal 7
- 8. Exposing service on Bare Metal (2) 8
- 13. Monitoring 13
- 16. 20
- 17. @apsega 2
Editor's Notes
- #5: No user modifications on live infrastructure = less human errors and outages. When new image is deployed, old one will be replaced, but not overwritten. In case of problems, older image can be deployed again. Container schedulers are always looking into declarative configuration and trying to keep the state as it is defined. Containers also provide isolation between applications running on the same machine, which helps avoid dependency conflicts. Application containers provide a clean abstraction for applications and when packaged in Docker image format, applications become easy to build, deploy and distribute.
- #7: Developers coming from Virtual Machines world tend to request much more resources than application needs. This is a big issue, when Instances are almost fully reserved, but CPU usage is quite low.
- #9: All k8s nodes under consul dns (0 TTL with health checks) Send-proxy-v2 for http and https endpoints
- #10: Separate VMs for etcd/api server/haproxy Physical machines k8s node machines
- #11: In multi-zone pods being deployed randomly
- #12: 3x replicas are being deployed in every zone