CDN algos
Download as PPTX, PDF4 likes761 views
The document discusses various algorithms used in content delivery networks (CDNs) including: 1) The Gale-Shapley algorithm for stable allocation of clients to server clusters. 2) Consistent hashing to balance load within clusters and cache popular objects on multiple servers. 3) Bloom filters for cache summarization and filtering one-hit wonders from caching. 4) Multi-commodity flow modeling for overlay routing between edge servers and origins to provide high availability and performance.
![Implementation challenges
• Unequal number of map units and clusters
• 10million map units vs 1000 clusters
• Partial preference lists
• To optimize we filter preference lists, for example, map
unit Boston does not need to have clusters from Tokyo in
it.
• Resource trees
• Used to capture cluster resources in Tree structure
• More
• Load-Balancing in Content-Delivery Networks [ppt]](https://image.slidesharecdn.com/27de8026-9a04-4972-920e-c8097e8d9e4e-160718071626/85/CDN-algos-7-320.jpg)
CDN algos
- 1. Algorithms used for CDN Hikmat
- 2. Introduction • Published in July, 2015 • by Bruce Maggs • Vice President of Research for Akamai • Professor at Duke University • Theoretical + Practical use of Algorithms for CDN • Link
- 3. Overview • Authoritative name server chooses Cluster • Stable allocation • Next, Server(edge) chosen in cluster (DNS ends here) • Consistent hashing • Which objects to cache and which not • Bloom filter • Find best route from edge to origin • Multi-commodity flow problem • Fault-tolerance for failures to make resilient system • Leader election algorithm
- 4. Stable allocation • Global load balancing is the process of mapping clients to the server clusters • Clients are represented as Map unit tuple <Client IP Address prefix, Traffic type> • For example: <1.2.3.4/24, video>, <1.2.3.4/24, web> • Properties: Demand(flash crowd), Traffic type, IP range • Cluster is set of servers • Usually Cluster is located in certain ISP or Autonomous system • Properties • Latency, packet-loss, throughput, capacity (CPU, RAM..).. • Task of Global load balancing is to match MapUnit -> Cluster • Each Map Unit has preferences for cluster in order • Example: <1.2.3.4/24,video> =>C1, C2, ...CN • Each Cluster has preferences for map units • Exampe: C1 => <1,2,3,4,video> as C1 is deployed in ISP where 1.2.3.4 is local • Algorithm used: "Gale-shapley" (next slide) How mapping should look
- 5. Gale-shapley algorithm • Introduced by Gale and Shapley in 1962 • Applied to "Stable marriage problem" • Find the most stable marriages for N men and N women • Input: Each man tells his/her preference by order • Ex: Nick -> Judy, Mary, Suzy.. • Output: Stable N matches (Man-Woman) • Reference: • WIKI • Code example • Details from Lecture Notes • Youtube animation • How it works in Next slide
- 6. How it works? Preferences of Men and Women 1. All men propose to each women 2. Women accept by their preference Details 3. rejected m2 now proposes to w4 4. w4 rejects m4, accepts m2 5. now m4 proposes to next. And So on. Finally we have 4x4 match
- 7. Implementation challenges • Unequal number of map units and clusters • 10million map units vs 1000 clusters • Partial preference lists • To optimize we filter preference lists, for example, map unit Boston does not need to have clusters from Tokyo in it. • Resource trees • Used to capture cluster resources in Tree structure • More • Load-Balancing in Content-Delivery Networks [ppt]
- 8. Consistent Hashing • Consistent hashing is used to balance the load within a single cluster • an object is mapped to the next instance of a bucket (server) that appears in clockwise order on the unit circle • Flash crowd to single object • Map popular object to the next K servers (k is a function of the popularity) • Content provider's objects are grouped by serial number, and hashed to same bucket • Uses same connection for multiple contents of same web page. • A212.g.akamai.net • References: • WIKI • CODE • Research paper • Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web
- 9. cluster Consistent Hashing in practice Server 1 Server 3 Server 2 Server 4 Authoritative Name server Resolving name server Images.example.com DNS CNAME: a212.g.akamai.net Choose Cluster with Gale alg. <IP, G-general> Cluster A Consistent hashing Serial num:212 "212" 's Popularity K servers selected e.g. K=2 DNS response: Server1 ,server2 In random order
- 10. Bloom filter • Probabilistic Data structure designed to tell you, RAPIDLY and MEMORY efficiently whether an element is present in a SET or Not. • Price paid for efficiency: element is definitely not in the set or maybe in the set with probability. • Browser Example: Detect if given URL is malicious • Input: million Malicious URLs, size approx. 25MB • With 99% probability 25MB reduced to 1.14MB • DEMO • http://billmill.org/bloomfilter-tutorial/ • Use case • Weak password, Hbase, Cassandra, Bitcoin (wallet sync)..
- 11. How it works • Input data is hashed K times (with different salt) and stored in bitset of size N • K is roughly proportional to the ratio of the size of the hash table and the number of elements that you expect to be. • Calculator: http://hur.st/bloomfilter?n=1000000&p=0. 01 • Example (figure on the right) • K = 3 • Java implementation • Wiki • https://en.wikipedia.org/wiki/Bloom_filter
- 12. Bloom filter in CDN • Cache summarization • More space-efficient than storing a list of URLs • Squid example (http://www.squid-cache.org/) • Distributes cache summaries to nodes (digests) • When object is evicted from cache, we use: • "Counting Bloom filters" • Integer replacing bits. Increment / Decrement for Insert / Deletion • Cache filtering • "One hit wonders" • ¾ of web objects accessed only ONCE per TWO days • Cache on second hit rule • Avoids caching one-hit wonders • Cache only when accessed second time during certain period • Two bloom filters: Primary and Secondary • Primary hold recent, Secondary past objects
- 13. Benefits of Cache on second Hit • Experiment conducted by Research Scientist at Akamai • On 47 prod. Servers • Enable "Cache on second hit" between March 14 ~ April 24 • When Cache On Second Hit is ON • Hit Rate increased from 74% to 83% (right figure) • Disk Usage decreased by 44% (left figure)
- 14. Benchmarking Bloomfilter with HashSet • Testing 1000000 elements add(): 1.204s, 830564.7840531562 elements/s contains(), existing: 0.919s, 1088139.281828074 elements/s containsAll(), existing: 0.934s, 1070663.8115631691 elements/s contains(), nonexisting: 0.867s, 1153402.5374855825 elements/s containsAll(), nonexisting: 0.914s, 1094091.9037199125 elements/s • Testing HashSet 1000000 elements add(): 0.226s, 4424778.761061947 elements/s contains(), existing: 0.081s, 1.234567901234568E7 elements/s containsAll(), existing: 0.063s, 1.5873015873015873E7 elements/s contains(), nonexisting: 0.195s, 5128205.128205128 elements/s containsAll(), nonexisting: 0.058s, 1.7241379310344826E7 elements/s
- 15. Overlay routing • Most of web objects are non-cachable or cachable only for short time • Short-term cache • Stock chart, weather data • Non-cachable: • Live streaming, Private data, Tele conference • We need algorithm to Construct overlay to provide efficient communication between edge servers and origin • Input: • Client demand: Edge -> origin, • Real-time network measurements between servers (latency, loss, bandwidth) • Output: Set of paths with high availability and performance How overlay routing looks
- 16. Multi-commodity Flow • The multi-commodity flow problem is a network flow problem with multiple commodities (flow demands) between different source and sink nodes. • Key Aspects for modeling • Multipath transport • Live video mode (replicate & collect & recover) • Web delivery • Role of overlay servers • Replicate, encode/decode • Cost function • For Dynamic web content "Latency" cost important • For Live stream "Throughput". • Capacity • Server / Network resources • Optimized transport protocols •TCP / IP optimizations
- 17. Algorithmic Solutions • Dynamic web content • All-pairs shortest path problem • Task: Select path with the best performance without capacity violations • Graph: Floyd-Warshall algorithm • Reference: • Network Flows: Theory, Algorithms and Applications • Live videos • Mixed integer problem • Reference: • Designing Overlay Multicast networks For Streaming • Algorithms for Constructing Overlay Networks for Live Streaming
- 18. Benefits of Overlay routing • Experiment 1 (Consistent benefits) • Origin server in Dallas, U.S, NA (North America) • Agents deployed around the world • Uncacheable content 38kb • Experiment 2 (Catastrophe Insurance) • In 2010 April, Internet outage SEA-ME-WE 4 • Cable was repaired from Apr.25~Apr.29 • Origin in Boston, Agents in Asia Figure1. Figure2.
- 19. Leader Election (nmon case) • Key concepts that underlie leader election • Failure model • Candidate set • Health • Electoral process • Similar to nmon (broadcasting health to others) • Outcome requirements • At-least-one, at-most-one
- 20. Used Algorithm • Raft !! (alternative to Paxos) • Consensus & Leader election algorithm • Awesome explanation • Github
- 21. Future work • Experimental • Cache on second-hit Challenges • Regression cases • Can be fixed with Custom header : X-Cache-On-First-Hit • TE-Chunked, Pseudo-chunked • Only disk • Should be applied as patch, not SAM