Advanced Caching Patterns used by 2000 microservices - Code Motion
- 1. Advanced Microservices Caching Patterns Natan Silnitsky - Backend Infra TL, Wix.com November 30, December 1-2, 2021
- 2. @NSilnitsky @NSilnitsky Caching Patterns Advanced Microservices Caching Patterns Natan Silnitsky Backend Infra TL, Wix.com natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
- 3. @NSilnitsky Caching Patterns >200 Million registered users from 190 countries
- 4. @NSilnitsky @NSilnitsky Caching Patterns Rebecca, Wix Stores Stores Checkout Service * issues
- 5. @NSilnitsky @NSilnitsky Caching Patterns 1. Get app identification 2. Start WIX APP MARKET Rebecca, Wix Stores HTTP Stores Checkout Service
- 6. @NSilnitsky @NSilnitsky Caching Patterns WIX APP MARKET Rebecca, Wix Stores 1. Get app identification 2. Start Stores Checkout Service Unreliable Requests * still available
- 7. @NSilnitsky @NSilnitsky Caching Patterns Alex, Wix Stores Store Catalog Stores Catalog Service retrieve
- 8. @NSilnitsky @NSilnitsky Caching Patterns Alex, Wix Stores Store Catalog Stores Catalog Service Request Overload DB latency and failure increase
- 9. @NSilnitsky @NSilnitsky No Cache - Increase in - loading and response - Network costs - DB failure Caching Patterns * traffic grows
- 10. @NSilnitsky @NSilnitsky With Cache - - reduce latency - reduce the needed scale - improve reliability Caching Patterns * DB is unavailable
- 11. @NSilnitsky When & how to Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. * S3 backed
- 12. @NSilnitsky When & how to Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. #2 To improve average latency for data layer access DynamoDB + CDC + LRU cache.
- 13. @NSilnitsky When & how to Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. #2 To improve average latency for data layer access DynamoDB + CDC + LRU cache. #3 Some very high external traffic cases Caches are very important - cache before the service.
- 14. @NSilnitsky When & how to Cache Examples from 2000 microservices of various use cases Caching Patterns #1 high risk/cost of network failure Like caching external critical configuration data. #2 To improve average latency for data layer access DynamoDB + CDC + LRU cache. #3 Some very high external traffic cases Caches are very important - cache before the service. #4 When NOT to - young products Don’t cache prematurely
- 15. @NSilnitsky Agenda 3 Caching Patterns: 1. S3 backed static cache 2. DynamoDB+CDC based cache 3. HTTP Reverse Proxy cache Caching Patterns
- 16. @NSilnitsky Agenda 3 Caching Patterns: 1. S3 backed static cache 2. DynamoDB+CDC based cache 3. HTTP Reverse Proxy cache Caching Patterns
- 17. @NSilnitsky Service WIX APP MARKET Caching Patterns for Unreliable Requests
- 18. @NSilnitsky Service WIX APP MARKET Caching Patterns On Init - Fetch app defs Koboshi / Remote Data Fetcher
- 19. @NSilnitsky Service WIX APP MARKET Caching Patterns Koboshi / Remote Data Fetcher Local cache or AWS S3
- 20. @NSilnitsky Service WIX APP MARKET Caching Patterns Koboshi / Remote Data Fetcher Local cache AWS S3 or + Retry
- 21. @NSilnitsky @NSilnitsky Caching Patterns Cache Type Cache Size Best For Not For Read-through cache Before the 3rd party service Data has to fit into memory External source of static configuration Dynamically updating data
- 22. @NSilnitsky Next 3 Caching Patterns: 1. S3 backed static cache 2. DynamoDB+CDC based cache 3. HTTP Reverse Proxy cache Caching Patterns
- 23. @NSilnitsky Caching Patterns Store Catalog Stores Catalog Service for Request Overload Read + Writes Browser HTTP * slower interaction
- 24. @NSilnitsky Caching Patterns Stores Catalog Service Cached KV Store 2. put Store Catalog 1. Write V1 K1 V1 K2 K3 K8 K9 V1 V2 V1 head End
- 25. @NSilnitsky Caching Patterns Stores Catalog Service Cached KV Store 1. get Store Catalog 2. Read on miss V1 K1 V1 K2 K3 K8 K9 V1 V2 V1 head End
- 26. @NSilnitsky Caching Patterns Stores Catalog Service Cached KV Store LRU Cache put Amazon DynamoDB put
- 27. @NSilnitsky Caching Patterns Stores Catalog Service Cross-DC Data replication & conflict resolution Amazon DynamoDB Cache
- 28. @NSilnitsky Caching Patterns Stores Catalog Service Amazon DynamoDB 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 Event K1- V2 is updated 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 K1 ---- V1 K2 ---- V1 K1 ----- V2 K3 ---- V1 K4 ---- V1 K5 ---- V1 CDC Topic Keeps a SEQUENCE OF EVENTS Kafka Broker Cache
- 29. @NSilnitsky Caching Patterns Stores Catalog Service Amazon DynamoDB 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 Event K1- V2 is updated 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 0 1 2 3 4 5 K1 ---- V1 K2 ---- V1 K1 ----- V2 K3 ---- V1 K4 ---- V1 K5 ---- V1 CDC Topic Keeps a SEQUENCE OF EVENTS Kafka Broker Consumer (Consumes from) Cached KV Store V1 K1 V1 K2 K3 K8 K9 V1 V2 V1 head End
- 30. @NSilnitsky Cache Caching Patterns Warm up Take most recent events Amazon DynamoDB CDC Topic Kafka Broker Stores Catalog Service
- 31. @NSilnitsky Cache Caching Patterns Warm up (compact) Take ALL Up to date events Amazon DynamoDB CDC Topic Kafka Broker Stores Catalog Service
- 32. @NSilnitsky @NSilnitsky Caching Patterns Cache Type Cache Size Best For Not For Cache-Aside Before the DB No limitation on DB data size LRU cache size is configurable Make DB read access faster for “popular” values Highly critical application startup information
- 33. @NSilnitsky Next 3 Caching Patterns: S3 backed static cache DynamoDB+CDC based cache HTTP Reverse Proxy cache Caching Patterns
- 35. @NSilnitsky Caching Patterns Site Renderer Service DNS resolution Load Balancers * simplified flow Wix DC Varnish cache * open source
- 36. @NSilnitsky Caching Patterns DNS resolution Load Balancers * simplified flow Varnish cache Kafka Broker Site Published Purge request (SiteId) Site Editor Service Site Renderer Service Wix DC
- 37. @NSilnitsky Caching Patterns DNS resolution Load Balancers * simplified flow Varnish cache Kafka Broker Site Published Purge request (SiteId) Purge ETag: SiteId Site Editor Service Varnish Purger Service Site Renderer Service Wix DC
- 38. @NSilnitsky @NSilnitsky Varnish cache Size limit depends on the configured Storage backends: 1. malloc is memory based — so the size is limited by (more) expensive memory cost 2. file is memory-backed-by-disk based — which is limited by less expensive disk cost, but can cause performance penalty due to fragmentation… Caching Patterns
- 39. @NSilnitsky @NSilnitsky Varnish cache Size limit depends on the configured Storage backends: 1. malloc is memory based — so the size is limited by (more) expensive memory cost 2. file is memory-backed-by-disk based — which is limited by less expensive disk cost, but can cause performance penalty due to fragmentation… 3. Massive Storage Engine (MSE) — also backed by file, but has a “fragmentation proof algorithm.” part of Varnish Cache Plus (paying customers only). Caching Patterns
- 40. @NSilnitsky @NSilnitsky Caching Patterns Cache Type Cache Size Best For Not For Read-through before the app Depends on storage backend Improving latency for “stable” HTTP responses Hard to invalidate aggregated data
- 41. @NSilnitsky @NSilnitsky Caching Patterns Choose Cache Pattern Is your data crucial for startup? Is your data highly dynamic? No Yes S3 Persisted data cache
- 42. @NSilnitsky @NSilnitsky Caching Patterns Choose Cache Pattern Is your data crucial for startup? Is your data highly dynamic? S3 Persisted data cache No Yes Is your data highly dynamic? Yes No DynamoDB backed cache / Redis Yes
- 43. @NSilnitsky @NSilnitsky Caching Patterns Choose Cache Pattern Is your data crucial for startup? Is your data highly dynamic? S3 Persisted data cache No Yes Is your data highly dynamic? Yes No External clients & easy invalidation? No DynamoDB backed cache / Redis Varnish cache / CDN Yes No Yes
- 45. @NSilnitsky A Java/Scala high-level SDK for Apache Kafka. github.com/wix/greyhound Caching Patterns 0.2 is out!
- 46. @NSilnitsky @NSilnitsky Thank You! 👉 slideshare.net/NatanSilnitsky Any questions? natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil Caching Patterns
- 47. @NSilnitsky @NSilnitsky Q&A natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil 👉 slideshare.net/NatanSilnitsky Any questions?