HTTP at your local BigCo
- 1. HTTP at your local BigCo:How the internet sausage gets made Peter Griess@pgriess
- 2. Goals and non-goalsBasics of TCP/IP, DNS and HTTP and how they work together; pitfalls and optimizationsA 1,000 foot view of scaling out HTTP infrastructureAll manner of load balancing / traffic shapingLiving on the edgeNot: how to make a fast application (database access, rendering performance, etc)
- 3. Background: DNSMap hostnames to IP(s)www.facebook.com 69.171.229.12, 69.171.228.40Resolution processRecursion (and what does the DNS server see?)CachingLatencies: on-host, cached in LAN, cached at ISP, miss
- 4. Background: TCPStateful protocolNegotiated by a synchronous 3-way handshake:2xRTT before first byte is sent!e.g. USA => South America ~250ms RTTSeamless failover is hard (but not impossible)Load balancing must be aware of flows
- 5. Background: HTTPLayered on top of TCP/TLSHas some useful bitsCompressionConnection re-usePipeliningCachingKind of sucksHeaders on all requests/responsesCompression on bodies onlyPipelining has to be disabled most of the timePipelining suffers from head-of-line blocking
- 7. Problem?
- 8. ProblemAvailabilityServer goes down (kernel panic?)Network goes down (cable cut?)Datacenter goes down (EC2?)OverloadShed load (good, can be transparent)Get infinitely slow (not good)
- 9. mycutekittens.tv: multi-serverBig bad internet???
- 10. We have optionsDNS load balancingIP load balancingHTTP load balancing
- 11. DNS load balancingmycutekittens.tv resolves to IPs: A, B, C, DAdd new IPs to scale outRemove IPs when hosts go downBenefitsDon’t need extra hardware to do load balancingCan span datacentersDNS servers are cheap / fastDrawbacksHotspots due to cachingHotspots due to ordering in result listHotspots due to resolver sizeTTL / flexibility trade-off
- 12. mycutekittens.tv: DNSBig bad internetDNS ServerDNS68.193.17.468.193.17.568.193.17.6
- 13. IP load balancing (1)mycutekittens.tv resolves to 1 public IP owned by an IP load balancerAdd new backend hosts w/ private IPs to scale outLoad balancer health-checks hosts actively or passively to avoid dead hostsScheduling policies vs. failoverDSR
- 14. IP load balancing (2)BenefitsOnly 1 public IP (high DNS TTL)Backend network capacity/membership transparent to the internetCheap-ishFailover is possible, not insanely difficultDrawbacksCan’t do what you can with HTTP
- 15. mycutekittens.tv: IP10.0.0.1Big bad internet10.0.0.2GW68.193.17.410.0.0.3LB
- 16. HTTP load balancing (1)mycutekittens.tv resolves to 1 public IP owned by an HTTP load balancerLargely same as IP load balancingTerminates TCP connections (sees all bytes)Can make routing decisions based on HTTPCan autonomously serve requests (caching, access control, etc)Examples:Send requests for /foo/* to pool A401 requests without cookie Q
- 17. HTTP load balancing (2)BenefitsLargely the same as IPMore flexible rulesCan terminate TLS (security+, cost+)DrawbacksNo DSRFailover difficultNot as performant as IP
- 18. mycutekittens.tv: HTTP10.0.0.1Big bad internet10.0.0.268.193.17.4LBHTTP(S)10.0.0.3
- 19. mycutekittens.tv: MOAREventually a single LB is going to be a problemNot enough capacityAvailabilityTurtles all the day way downLB of LBs!DNS load balancing between datacenters…
- 20. HTTPS: myths and realityToo computationally expensiveOnly a few percent (imperialviolet.org); is your webserver actually CPU bound? doubt itSSL acceleration cards, GPUs, etcToo much latencyHandshaking is 5-7xRTTSession resumeFalse startSnap startCaching breaks
- 21. My latency is huge in JapanRTT to USA is (or any single DC) can be hugeRe-use connections (connection: keep-alive)Send work in parallel (pipelining)Use compression (content-encoding)Lots of tricks for static resources (bundling, CDNs, caching, etc)Pre-fetch data
- 22. Let’s get crazy: SPDYDon’t limit yourself to HTTP; use a different protocolSPDY developed by Google, supported by Chrome, google.com (and soon facebook.com)Connection re-use w/o head-of-line blockingHeaders always compressedAlways SSL (but breaks caching)
- 23. Let’s get crazy: TCP terminationSynchronous RTTs: the silent killerOpening new TCP connections is very costlyRun proxies close to users and proxy traffic back to core using optimized protocolLow RTT to proxyDo SPDY-like tricks between edge + corePotentially faster network to core than public internetAdvertise these proxies via DNSGeo-targettingAS-adjacencyAkamai CDN does this, sort of
- 24. Let’s get crazy: DNS anycastRemember how DNS resolutions were slow?DNS servers could be far away from a userAdvertise multiple network routes for the same DNS IP, let the IP stack pick the closest one