Traffic Control with Envoy Proxy
- 1. Better Traffic Control with Envoy Mark McBride 1/31/2018
- 2. Why Care about Traffic Control Generaliza5ons: Kubernetes leads to a bunch of good things. • Crea5ng new services is easier. • Deploying new service versions is easier. • Deploying smaller services is easier.
- 3. Why Care about Traffic Control But the good things aren’t free. • New code needs to be (safely!) integrated with your request flow. • Addi5onal abstrac5ons have < 100% reliability. • Longer call chains introduce more chances for failure.
- 4. Goals of Traffic Control • Resilience • Distributed systems are never “up”1. Dealing with failures should be straighTorward. • Rou5ng • Introducing a new code to the call chain is a common opera5on. It should be straighTorward. 1. Charity Majors, hYps://opensource.com/ar5cle/17/7/state-systems-administra5on
- 5. The Setup • Create scenarios using augmented Envoy examples • Use wrk to drive load against the system and measure results • Curl, because no demo is complete without some curl • A preview of envoy-tools to observe Envoy stats directly
- 6. Control Requires Visibility • Making unobservable changes is not advised. • Envoy comes with great tools out of the box. • Stats on listeners, clusters, protocols, and more. • An admin server for direct observa5on and control. • envoy-tools (coming soon!) – a repository of tools that provide a more approachable interface.
- 7. The Examples
- 8. Adding Reality to Examples • Add configurable latency and success rate
- 9. Adding Reality to Examples
- 10. Retries Envoy supports retry policies aYached to routes • Select error codes to retry on. • Configure 5meouts for each retry. • Configure number of retries.
- 12. Retries—a Closer Look Failures when calling service1 No failures returned to client
- 13. Safe Retries • Usually you don’t want to retry all requests. • Side effects are important to consider. • Atomicity is important to consider. • Computa5onal expense is important to consider. • Add more routes, and configure retries accordingly.
- 14. Load Shedding • Some5mes you get more traffic than you can handle. • Envoy supports request limits on a per-cluster basis. • Envoy also supports two priority groups, allowing you to save slots for important traffic.
- 15. Without Circuit Breakers Failures are fine, but 99% latency is slowwwwww as requests just back up Also, POST requests are totally offline because we’re swamped with GETs
- 16. Without Circuit Breakers Retries overflow, which is slow
- 17. With Circuit Breakers Gobs of failures, but p99 latency is s5ll good. Also, POST requests are available. Also, we told clients to back off with the x-envoy-overloaded response header.
- 18. With Circuit Breakers Pending requests overflow, which is fast!
- 19. An Overview of Rou5ng • Endpoint metadata for richer rou5ng primi5ves • Probabilis5c distribu5on of traffic across mul5ple clusters • 1% of traffic to my-great-rewrite, 99% to legacy • 1% of traffic to v2 of my service, 99% to v1 • Header based rou5ng to cluster subsets • If “x-canary” is set route to endpoints with a version label of v2 • Priority rou5ng, which we saw in the circuit breaking example • Zone aware rou5ng
- 20. Traffic Shioing the Hard Way • Mul5ple clusters • Mul5ple routes
- 21. Header-based Canary • When we specify the canary header, the route matches and we (and only we) are routed to service1a • When header is not present, the route doesn’t match and we go on to the next route, sending traffic to service1
- 22. Probabilis5c Rollout • With the run5me match, we choose this route 25% of the 5me, sending 25% of our traffic to service1
- 23. Traffic Shioed 25% of traffic to service1a
- 24. The Easy Way • Restar5ng servers on every config change is tedious in this demo. • It’s even more tedious in produc5on. • Envoy provides a beYer way—the xDS APIs.
- 25. xDS APIs • CDS - discover clusters, which are logical groupings of endpoints. • A cluster defini5on can have a reference to an EDS endpoint • EDS - discover endpoints for a cluster. • LDS - discover listeners for an Envoy • A listener’s filter chain can have a reference to an RDS endpoint • RDS - discover routes for a filter chain
- 26. Dynamic Config • The xDS APIs give you a central point-of-control to manage a fleet of Envoys • Bridge service discovery (e.g. from Kubernetes) to Envoy • Bridge rou5ng config (e.g. from Houston) to Envoy
- 27. Advanced Rou5ng with EDS • CDS (cluster discovery service) defines groups of endpoints. • EDS (endpoint discovery service) discovers the actual endpoints for clusters. • EDS allows you to aYach metadata to an endpoint. • Our mul5-cluster example can be collapsed to a metadata based approach on a single cluster.
- 28. Even Easier with Houston • An CDS/EDS server with integra5ons to EC2, ECS, Kubernetes, Consul, DC/OS, or JSON files • An LDS/RDS server with an intui5ve route configura5on UI • Stats parsing, forwarding, and change tracking
- 29. Ques5ons/Contact Mark McBride mark@turbinelabs.io Twitter - @mccv http://www.turbinelabs.io