Evolving Your Distributed Cache In A Continuous Delivery World: Tyler Vangorder
0 likes218 views
1. The document discusses the evolution of caching strategies at Build.com as their systems and traffic grew rapidly over time. They initially used a Java-based distributed cache and later switched to Redis which proved more effective. 2. As Build.com moved to a continuous delivery model with multiple environments, they needed a "shared" cache that both environments could use. They implemented a unified caching model where each version of code has its own bucket in the cache but objects can be promoted from older versions if they are compatible. 3. The key aspects of the unified caching model are using a serialization checksum to detect changes between versions, using a build number as the cache key so each version is separate, and attempting to promote
Evolving Your Distributed Cache In A Continuous Delivery World: Tyler Vangorder
- 1. Evolving Your Distributed Cache In a Continuous Delivery World Tyler Van Gorder @tkvangorder Build.com, Principal Software Engineer
- 2. PRESENTED BY 1 The Early Days Who we are and how our systems have evolved 2 Today’s Architecture A peek into our current system architecture 3 Unified Caching Model (Live Coding Example) Solving Caching Problems in a Continuous Delivery Environment Agenda: 4 Questions
- 3. A personalized, online, home improvement shopping experience Tyler Van Gorder Principal Software Engineer https://github.com/tkvangorder @tkvangorder
- 4. BUILD.COM AT A GLANCE 109K Professional Tradespeople 91M Annual Visits >569K Annual Customer Chats >1.3M Customer Calls Annually #72 Top 100 Internet Retailers in the US 710+ Employees
- 5. PRESENTED BY The Early Days
- 6. Started as a single web site The site and company grew rapidly Even the CEO helped with coding (oh dear) Deployment were done at night Performance issues
- 7. There were several distributed caching options We tried a Java-based distributed cache first We decided to use Redis Caching? Round 1 - Fight! https://pixabay.com/users/12019-12019/?utm_source=link-attribution&utm_medium=referral&utm_campaign=image&utm_content=115800
- 8. It’s hard to argue with the results Why Redis? Redis can be clustered Redis is blazingly fast Easy to maintain/upgrade Built-in support in for Spring.
- 9. Introducing a cache proved very effective. We did deployments at night Deployment meant purging the cache It worked for a while And we continued to grow Caching To Rescue https://pixabay.com/vectors/superhero-super-hero-hero-figure-304713 Distributed Cache!
- 10. Multiple web storefronts We put more data into cache Sites were really slow after deployment We introduced cache priming Held our breath as morning arrived We need to fix this! We Are Killing Ourselves Story Time https://pixabay.com/vectors/superhero-super-hero-hero-figure-304713
- 11. Two identical environments (prod./dark) Both environments use the same data stores Deploy to the dark servers in the day Still had to purge and prime the cache Blue / Green Deployments
- 12. https://pixabay.com/vectors/superhero-super-hero-hero-figure-304713 Production traffic continued to grow We still had two different redis clusters Internal canary Difficult to move to continuous delivery Story Time We Need a “Shared” Cache
- 14. Consumer Store Nodes Mobile API Internal Tools Platform API Services System Architecture
- 15. Data Access Common Business Logic Security Caching! Platform API Image by PIRO4D from Pixabay
- 16. Load Balancing System Metrics Log Aggregation Twelve-Factor Applications Circuit breakers Rolling deployments https://12factor.net/
- 17. Push code to production often Changes are smaller/less risk Immediate user feedback Continuous Delivery Automate the process Monitor for issues Need the ability to rollback https://harness.io/
- 19. The Trouble Spots In Rolling Deployment System must be resilient to having two versions of your application running concurrently. Both versions of the application need to play nice when they are sharing a cache.
- 20. PRESENTED BY Unified Caching Model
- 21. The application is caching customer data Version 1 has basic customer object Version 2 adds address information to the customer Demo Setup Round 2 - Fight (and a demo?) https://pixabay.com/users/12019-12019/?utm_source=link-attribution&utm_medium=referral&utm_campaign=image&utm_content=115800 https://github.com/tkvangorder/redis-shared-cache-sample
- 22. What can go wrong? https://github.com/tkvangorder/redis-shared-cache-sample
- 24. V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put V1.0 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put V1.1 : Cache Get, Cache Hit, Checksum Failure, Retrieve, Cache Put Cache Thrashing Insanity: Doing the same thing over and over again and expecting different results. - Albert Einstein https://github.com/tkvangorder/redis-shared-cache-sample
- 25. Cache Thrashing In The Wild https://github.com/tkvangorder/redis-shared-cache-sample
- 26. Caching Version Buckets https://github.com/tkvangorder/redis-shared-cache-sample
- 27. Cache Promotion Version cache miss, look for another version’s copy If the checksum is successful, copy is compatible Promote that copy into cache for the new version https://github.com/tkvangorder/redis-shared-cache-sample
- 28. PRESENTED BY 1 Use a serialization checksum The use of an incrementing uid for each complex object insures changes to nested objects are also detected. This requires a change in the development process! 2 Use a build number as your nested hash key Each active build will have it’s own version of a cached object. Automate the injection of this build number into your software artifacts! 3 Attempt a cache promotion on a cache miss. If a new build does not have a copy of the cached value, attempt to deserialize the previous version. If there are no structural changes, promote! Summary:
- 29. PRESENTED BY 1 What about cache eviction? The root key is still pinned to the record, eviction in one version will evict ALL copies. 2 Time to live helps manage the overall resource requirements Multiple copies of the same object requires more resources for you Redis cluster. In practice, we have found this to be a non-issue in our environment. 3 We have written the unified cache implementation twice We originally extended the default Redis cache implementation in Spring Boot 1.5.x, now we have our own cache implementation. Final Notes: 4 Open Source? All of the sample code, including the unified cache library demonstrated here are available for your use. If there is enough interest, we may make an official open source offering.
- 30. PRESENTED BY Questions? Tyler Van Gorder Principal Software Engineer https://github.com/tkvangorder @tkvangorder
- 31. Thank you!