Stackpath use case
1 like401 views
This document discusses using the vmod_kvstore module in Varnish to efficiently route requests for a content delivery network (CDN) with a million virtual hosts (vhosts). The traditional approach of using if/else statements in VCL does not scale well for this use case. vmod_kvstore allows storing the mapping of host headers to backends in an in-memory hash table for O(1) lookups. This avoids recompiling the large VCL and improves reload performance. Benchmarking showed vmod_kvstore could route requests for 20,000 vhosts much faster than traditional VCL. The key-value store also enables features like caching time overrides and alias mappings per vhost.
Stackpath use case
- 1. Scaling Varnish to Handle One Million Vhosts Nathan Moore
- 2. MaxCDN acquired by StackPath in 2016 • https://techcrunch.com/2016/07/25/security-as-a-service- startup-stackpath-nabs-180m-4-acquisitions-including-maxcdn/ 9/21/16 2
- 3. Content Delivery Networks – CDN FAQ 9/21/16 3
- 4. CDN is a global web proxy 9/21/16 4
- 5. CDN is a caching global web proxy 9/21/16 5
- 6. CDN is a caching global web proxy for a million VHosts 9/21/16 6
- 7. CDN has a HTTP object routing problem • How does CDN proxy for a million VHosts? 9/21/16 7
- 8. Routing decisions based on HOST header • Classic VCL If (req.http.host == “www.foo.com”) { set req.backend_hint = fooOrigin; } else if (req.http.host == “www.bar.com”) { set req.backend_hint = barOrigin; } … 9/21/16 8
- 9. Classic VCL with if…then statements • Require 1 million “if else” statements • varnish reload times excessive • Must recompile VCL to reload dynamically • If latency to recompile VCL is large, can exceed reload timeout 9/21/16 9
- 10. Quick benchmark • Executed on local laptop in a single CPU VM to emphasize latency difference. • Using h2load run locally to request a 100B object • h2load -n 50000 -c 1 -m 1 --h1 http://h1.netdna-dev.com:6081 • Directly to origin on localhost: 9/21/16 10
- 11. Quick benchmark • Through Varnish and its if…then statements (only 2500 in this case!) 9/21/16 11
- 12. Quick benchmark • Through Varnish and its if…then statements (only 3 in this case!) 9/21/16 12
- 13. Enter vmod_kvstore • A general key-value store featuring O(1) lookups • In-memory hash • For this use case, HTTP object routing: • HOST header is the key • Backend server is the value 9/21/16 13
- 14. vmod_kvstore • Old VCL: If (req.http.host == “www.foo.com”) { set req.backend_hint = fooOrigin; } else if (req.http.host == “www.bar.com”) { set req.backend_hint = barOrigin; } … 9/21/16 14
- 15. vmod_kvstore • New VCL: Set req.http.backend_name = kvstore.get(0,req.http.Host,“defaultBackendName”); If (req.http.backend_name == “myBackend1”) { set req.backend_hint = myBackend1; } else { set req.backend_hint = defaultBackendName; } 9/21/16 15
- 16. Quick benchmark, updated • Through Varnish and vmod_kvstore defining 20000 VHosts 9/21/16 16
- 17. vmod_kvstore • Getter • kvstore.get(unique kvstore number, key, default value) • Can have multiple kvstores, each referred to by a unique number • Default value is returned if key is not in the kvstore keyspace • Setter • kvstore.set(unique kvstore number, key, value, ttl) • This sets a key->value pair • The ttl is a time-to-live in seconds • kvstore can be used to store temporary data 9/21/16 17
- 18. vmod_kvstore bulk create • kvstore.init_file(unique kvstore number, number of buckets, path to file, delimiter) • Predefined file consists of • req.http.Host -> req.backend_hint name values 9/21/16 18
- 19. vmod_kvstore dynamic update • Setter method allows dynamic update of kvstore • Use case: • Could use HTTP GET request with custom header to allow dynamic updating If (req.http.X-Value-AddToKvstore) { kvstore.set(0,req.http.Host,req.http.X-Value-AddToKvstore,0); } 9/21/16 19
- 20. More vmod_kvstore use cases • Map multiple aliases to a Vhost kvstore.set(0, ”content.foo.com”, ”fooVhost”); kvstore.set(0, ”static.foo.com”, ”fooVhost”); kvstore.set(0, ”fooVHost”, ”fooBackendHint”); • Now, workflow is: req.http.X-VHost = kvstore.get(0, req.http.Host, ”defaultVHost”); req.http.X-Backend = kvstore.get(0, req.http.X-Vhost, “defaultBackend”); If (req.http.X-Backend == ”fooBackendHint”) { set req.backend_hint = fooBackendHint; } 9/21/16 20
- 21. More vmod_kvstore use cases • Override caching time on a per-VHost basis kvstore.set(0, ”www.foo.com”, ”fooVHost”); kvstore.set(0, “fooVHost.cacheTime”, ”30s”); • Now, workflow is: req.http.X-VHost = kvstore.get(0, req.http.Host, ”defaultVHost”); set beresp.ttl = kvstore.get(0, req.http.X-VHost + “.cacheTime”, ”3600s”); 9/21/16 21
- 22. Conclusions • As more logic is pulled into Varnish, more is asked of Vcl • Make dynamic decisions based on any property of an HTTP object • Persist state within kvstore through loading from file • Dynamically update state within kvstore in-memory • Scale ability to make decisions across millions of possible keys and values • Gain flexibility in implementation 9/21/16 22
- 25. Partitioning backend servers • Can always pass through multiple proxies 9/21/16 25
- 27. Partitioning backend servers with Origin Shield 9/21/16 27