Loadbalancers: The fabric for your micro services
0 likes792 views
The document discusses the importance of load balancers for microservices and container-based architectures. It covers how load balancing needs have evolved from traditional monolithic applications to modern approaches that support dynamic microservices topologies, zero-downtime deployments, client-side load balancing, and advanced patterns like circuit breakers. It also introduces NetScaler CPX as a cloud-native load balancer solution for container environments.
Loadbalancers: The fabric for your micro services
- 1. LOADBALANCERS: THE FABRIC FOR YOUR MICROSERVICES Chiradeep Vittal API World 2016
- 2. About me ¨ Distinguished Engineer at Citrix ¨ Work on Citrix Netscaler and containers
- 3. Load balancers are important (again) ¨ Containers and Microservices ¨ Background on Load Balancing ¨ Teaching old load balancers new tricks ¨ Emerging patterns for Load Balancing in microservices ¨ Future directions
- 4. Information Technology is rapidly accelerating Time Web Digital Social Mobile Things Smart Things RateofChange
- 5. The challenge for IT Match the velocity of business change
- 6. Addressing the gap: Agile © Christopher Little / Wikimedia Commons / CC-BY-3.0
- 7. ¨ Collaboration between Dev and Ops teams ¨ Tooling and automation to squeeze commit-to-deploy time Addressing the Gap: DevOps
- 8. Agile and DevOps 1. Frequent Iterations 2. All phases all the time 3. Risk-based 1. Shorten time from commit to production 2. “You build it, You run it” 3. Measure everything 4. Experimentation and learning Agile DevOps
- 9. Agile + DevOps Revolution 10+ Deploys per Day Flickr, 2009 50 million Deploys Per Year Amazon Web Services, 2015
- 10. Image by BMW Werk Leipzig - http://bmw-werk-leipzig.de, CC BY-SA 2.0 de, https://commons.wikimedia.org/w/index.php?curid=11928438 Key Mechanisms: Automation
- 12. 1. Small Autonomous processes 2. Loose coupling 3. Bounded contexts 4. Deploy & scale independently Key Mechanisms: Microservices
- 13. Containers & speed Compile (seconds) VM Build / Configure (minutes) Deploy VM (minutes) Compile (seconds) Container Build (seconds) Container Deploy (seconds) Before Containers (virtual machines) With Containers
- 14. Container Wins ¨ Small & Autonomous ¨ Deploy and scale independently ¨ Increased Velocity ¨ Reduced risk ¨ Efficiency Features Benefits
- 15. Containers or Microservices? ¨ Microservices == architecture ¨ Containers == implementation ¨ Containers win over VMs for Microservices
- 17. Load Balancing: Traditional vs. Cloud Native Static Applications, well defined topology Dynamic Microservices, changing topology W W W A A A A A W M M TrafficismostlyN-S M M M M M M M M M Traffic is mix of N-S and E-W
- 18. Cloud-Native Patterns of architecture and organization that deliver software with speed and reliability ¨ Auto Scale ¨ Continuous Delivery ¨ Baked-in Resilience ¨ Deep Monitoring ¨ Collaboration
- 19. Cloud Native + Containers is Network Intensive ¨ Implications on ¤ Performance ¤ Reliability ¤ Security ¤ Routing ¤ Naming and discovery ¤ Monitoring
- 20. Cloud Native Landscape - Microservices Microservices APIs Performance Resilience Security Visibility Continuous Delivery AutoScal e Circuit Breaker Load Balancing Throttling Discovery Audit Segmentation E2E Encryption Routing Chaos Monkey Distributed Debug Back-off Lifecycle Management Auth
- 21. Containers Container Networking Container Security Integrity Isolation AuthN/Z Container Orchestration Infrastructure Orchestration Network Orchestration Cloud Orchestration Multi-cloud Orchestration Private Cloud OrchestrationHypervisor Orchestration Network Orchestration IPAM / DNS Overlay / Underlay Microservices Lifecycle Management Container Performance Network Perf scheduler Container Storage Storage Orchestration Storage Orchestration Cloud Native Landscape - Infrastructure
- 22. Container Managers to the rescue? Docker Swarm Kubernetes (from Google) Mesosphere DCOS (based on Apache Mesos) AWS ECS
- 24. Cluster Managers – Sweet Spots ¨ Infrastructure independence ¨ Scheduling ¨ Discovery ¨ Scaling (partially) ¨ Security (a little bit) ¨ Load balancing (limited)
- 25. A brief history of Load balancing Internet W W High Availability Internet W W WW HA + Load Balancing Internet Webserver
- 26. A brief history of Load balancing Internet W WW App 1 App 2 W WW foo.com/app1 foo.com/app2 L7 routing Internet W WW SSL Traffic Unencrypted Traffic Internet W WW App Firewall
- 27. What LB-in-the-wire enables ¨ Resilience ¨ Encryption offload ¨ Application acceleration ¨ Defend L4à L7 ¨ L7 routing ¨ Application Performance Analytics
- 28. Load Balancing Form Factors Hardware Software VM Virtual As-a-service Containerized
- 29. Physical Price-Performance MPX VPX Hypervisor Virtual Run Anywhere = NetScaler CPX: Cloud Native Load Balancer CPX (new) Container SDX Platform Multi-Tenant =
- 30. NetScaler CPX Containerized NetScaler for Developers / Cloud native deployments ¨ Same hardened code, same features* as bigger form factors ¨ Unified control plane with Ingress and other LB (NetScaler MAS) ¨ Unified monitoring, logging and analytics * Same as NetScaler Standard Edition minus VPN
- 31. NetScaler CPX Express ¨ “Developer” Edition* ¨ No license required ¨ Available for download from Docker App Store in Q3 ¨ Drop-in replaceable with licensed version ¨ Get it here: – http://microloadbalancer.com *Not for production use
- 33. Load Balancing for container clusters – Ingress / Edge α5 α6 HostH4 Scale out Internet α1 β1 δ1 HostH1 β2 α2 δ3 HostH2 α3 α4 δ2HostH3 Public Endpoint https://alpha:443 LB
- 34. Load Balancing – intra-cluster α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LB LB per endpoint
- 35. Load Balancing – intra-cluster α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LBLB LB LB per container host • Service clients use an LB local to their host • E.g., Kube-proxy, Swarm, Mesos - MinuteMan
- 36. Ingress vs Intra-cluster differences ¨ Performance ¨ Count ¨ Frequency of reconfiguration ¨ L4 (intra-cluster) vs L7 (ingress) ¨ Form factor
- 37. Reconfiguration of Load Balancer α1 β1 HostH1 β2 α2 HostH2 α3 α4 HostH3 Ingress LB α5 α6 HostH4 ClusterManager LB Controller Cluster API Container Events Reconfigur ation Container State Container State Query
- 38. Nitrox – Configure Citrix NetScaler for Cluster Managers ¨ github.com/chiradeep/nitrox ¨ Apache license ¨ Support for integrating NetScaler with ¤ Kubernetes ¤ Swarm ¤ Consul ¤ Marathon
- 39. Microservices LB patterns ¨ Client-side LB ¨ Zero-downtime deploys ¨ Resilience patterns ¨ Visibility / Insights
- 40. Client-side LB ¤ Embedded into calling application n e.g., Netflix Ribbon (with Eureka) ¤ Run as side-cars (alongside each application or one- per-host) Eureka M M M M M M
- 41. Client-side LB ¤ Run as side-cars, or one-per-host Service Discovery M M M M M M Clients Service LB LB
- 42. Client-side LB Examples ¨ Netflix Prana - Hystrix as side car (deprecated) ¨ Linkerd from Buoyant.io (based on Twitter Finagle) ¨ AirBnB Smartstack/Synapse (uses HAProxy) ¨ Uber Hyperbahn ¤ Like Finagle, switched from HAProxy ¨ Kube Proxy ¤ Initially user-space, now based on IPTables ¨ Docker Swarm LB ¤ Based on IPVS ¨ Mesos LB ¤ Based on IPTables
- 43. Zero-Downtime Deploys ¨ Canary deploys ¨ Traffic can be selected based on source (country), headers M M M M Version N M M M M M Version N+1 10%90% CD Pipeline
- 44. Zero-Downtime Deploys ¨ Blue-green deploys M M M M Version N M M M M M Version N+1 100% CD Pipeline
- 45. Zero downtime deploys with NetScaler HostH1 HostH2 HostH3 Ingress NetScaler HostH4 ContainerClusterCluster API Container Events Reconfig uration Controller NetScaler CPXJenkins
- 46. Advanced Resilience – Circuit Breaker ¨ Circuit Breaker is a pattern that prevents cascading failures due to unresponsive downstream services ¨ A load balancer can be configured to ignore a slow backend Credit: http://martinfowler.com/bliki/CircuitBreaker.html
- 47. Visibility ¨ Compliance ¨ Topology ¨ Analytics M M M M M M M M M M M Analytics
- 49. Wrap-up ¨ Load balancing is different for containers / microservices ¨ Integration with container cluster managers is needed ¨ Consider using the same LB technology for ingress and intra-cluster ¨ Emerging patterns solidify the importance of the load balancer ¨ LB in the wire brings ¤ Simplicity ¤ Resilience ¤ Future proofing