Optimizing kubernetes networking
5 likes1,351 views
The document discusses optimizing Kubernetes networking for large-scale environments handling trillions of data points and 1000-2000 node clusters. It addresses challenges such as latency, load balancing, and cross-cluster access while exploring various CNI plugins and ingress configurations for improved performance. Additionally, it highlights the limitations of current load balancer support on platforms like AWS and GCP, indicating ongoing challenges in achieving optimal networking solutions.
Optimizing kubernetes networking
- 2. Throughput Trillions of data points daily Scale 1000-2000 nodes clusters Network challenges Latency End-to-end pipeline Topology Multiple clusters Access from standard VMs
- 3. IPVS Native Load-balancer More efficient Still a bit young No Bridging Route on the host Specific CNI plugins Better Latency Less CPU usage Native pod routing Pod get standard IPs No overlay overhead Cross-cluster access Much better ingresses Addressing this in Kubernetes
- 4. On premise BGP Calico Kube-router AWS Additional IPs on ENIs AWS EKS CNI plugin Lyft CNI plugin GCP IP aliases Specific controller config Native pod routing
- 5. Master Ingress traffic External Client Load-Balancer pod pod pod kube-proxy kube-proxy kube-proxy NP NP NP Heathchecker data path health checks configuration (from watching ingresses on apiservers) service-controller
- 6. Master ExternalTrafficPolicy: Local? External Client Load-Balancer pod pod pod kube-proxy kube-proxy kube-proxy NP NP NP Heathchecker data path health checks configuration (from watching ingresses on apiservers) service-controller
- 7. K8s HTTP ingresses External Client Load-Balancer pod pod pod kube-proxy kube-proxy kube-proxy NP NP NP Heathchecker data path health checks ingress-controller configuration (from watching ingresses on apiservers) Drive HTTP Loadbalancers
- 8. L7-proxy ingress controller data path health checks configuration from watching ingresses/endpoints on apiservers (ingress-controller) from watching LoadBalancer services (service-controller) External Client Load-Balancer l7proxy l7proxy kube-proxy kube-proxy kube-proxy NP NP NP Heathchecker ingress-controller pod pod pod pod Create l7proxy deployments Update backends using service endpoints Master service-controller
- 9. With native pod routing External Client Load-Balancer pod pod pod Heathchecker data path health checks ingress-controller configuration (from watching ingresses/endpoints on apiservers) AWS: ALB with pod IP GCP: GCLB with NEG
- 10. Limited to HTTP ingresses No TCP/UDP traffic Need to change LB controllers (NLB / NEG support TCP) Limited Loadbalancer support No ELB No ILB Remaining challenges
- 11. Thank you Questions? @lbernail Datadog booth Also, We’re hiring