Varnishtest
1 like236 views
This document summarizes David Porter's talk about leveraging the power of Varnish Configuration Language (VCL) and Varnishtest. It outlines how the company improved their practices around traffic routing, caching, and testing of VCL configurations by adopting automation tools like Puppet and Jenkins for configuration management and continuous integration, using Docker for testing environments, and implementing a software development life cycle with peer review and version control for VCL files. This allowed them to move from a problematic, manual process to one where configurations are more robust, standardized across environments, and collaboratively developed and tested.
Varnishtest
- 1. LEVERAGING THE POWER OF VCL AND VARNISHTEST David Porter Tech Lead / Software Developer – Global eCommerce Varnish Summit Boston – March 2017
- 2. PURPOSE OF THIS TALK To inspire VCL users to embrace software development practices for all to benefit.
- 3. THE PLAN• Where we were • Where we are
- 4. WHERE WE WERE F5s, NetSec, Tickets, Trouble
- 5. And it was not good. 5 • F5 Local Traffic Manager (LTM) • Used as a load balancer, ACL, and a bit more… • It's software (TCL) but no development practices. ‣ All handled by NetSec, tickets only, super slow ‣ Prone to typos, no linting ‣ No version control, full copy/paste, rapidly growing files ‣ No testing or measure to prevent regression ‣ Not much of a review process (code pasted in ticket) ‣ Too much duplication between environments, nothing shared In the beginning was the F5…
- 6. A lot more. 6 • Broaden the team and contributor base ‣ Insight into logic ‣ Insight into configuration ‣ Not just netsec/sysops/devops • More sophistication ‣ LTM… Varnish is an LTM/LB too, but cooler ‣ Caching… Can’t live without it ‣ HTTPhandling...Tune your requests/responses We needed more.
- 7. WHERE WE ARE Traffic Routing, HTTP handling, Caching
- 8. 8 • Routing by hostname ‣ e.g. status.rackspace.com • Routing by path ‣ e.g. rackspace.com/knowledge_center • Environment / ACL handling ‣ Internal URLs ‣ Admin Panels • Long-term redirection ‣ Old hostnames ‣ Old resources Local Traffic Management (LTM)
- 9. 9 • CORS (Cross-Origin Resource Sharing) ‣ Global policy with overrides from backends • HTTP methods ‣ e.g. preventTRACE method • Normalization ‣ Fix wacky requests ‣ e.g. rackspace.com///////////cloud • Framebusting ‣ Global policy ‣ Hostname-specific exceptions (e.g. VWO heatmap tool) • ESI ‣ When and where appropriate HTTP Handling
- 10. 10 • 90%+ hit rate ‣ e.g rackspace.com/cloud?foo=bar -> rackspace.com/cloud ‣ Path-specific overrides ‣ Disregard query params by default ‣ Internal overrides only • Grace ‣ Invaluable during downtime ‣ Great during maintenance periods / updates Caching
- 11. ALL TESTED Very satisfying, gains confidence.
- 12. HOW WE GOT HERE Automation, Continuous Integration, SDLC
- 13. 13 • Puppet ‣ Configuration automation / simplification ‣ ERB templates • Jenkins CI ‣ Puppet + Docker = great for idempotent behavior ‣ Run/lint Puppet-generated VCL ‣ Run Varnish tests against latest VCLin Docker container • Ant ‣ TaskAutomation ‣ Defined set of tasks, no checklists for humans Automation, automation, automation
- 14. 14 • Local ‣ Vagrant VMs use same Puppet configs as staging/production • Docker ‣ Temporary environments for CI ‣ Simplified CI architecture for all applications • Full Staging Environment ‣ Comprehensive replica of production, top to bottom ‣ Smaller than production, still uses HAtechniques (multiple machines) • Active/Passive Production ‣ Call it pre-prod ‣ Multiple production envs = better disaster recovery (DR) Environments
- 15. 15 • One repository to rule them all • Better than just documentation • Developers can contribute ‣ Path for learning VCL ‣ Foundation in varnistest • DevOps engineers can contribute ‣ Path for learning VCL ‣ Foundation in varnistest • Peer Review ‣ At least two peers required to sign-off ‣ Preferrably 1 DevOps, 1 Developer SDLC (Software Development Life Cycle)
- 16. EVERYBODY WINS No one is alone, nothing is lost.
- 17. CLOSING REMARKS To inspire VCL users to embrace software development practices for all to benefit.
- 18. THANK YOUContact me – david.porter@rackspace.com See me afterward for show and tell