Where is my cache? Architectural patterns for caching microservices by example
- 1. Where is my cache? Architectural patterns for caching microservices by example Rafał Leszko @RafalLeszko Hazelcast
- 2. About me ● Cloud Software Engineer at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
- 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Hiring (Remote)! ● Recently Raised $21M ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com
- 4. ● Introduction ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 5. Why Caching? ● Performance ○ Decrease latency ○ Reduce load ● Resilience ○ High availability ○ Lower downtime
- 6. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World
- 7. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World cache cache cache cache
- 8. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World cache cache cache
- 9. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World cache cache cache
- 10. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 11. 1. Embedded
- 13. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 14. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 15. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!
- 17. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();
- 18. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }
- 19. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!
- 22. Application Application Load Balancer Cache Cache Request Hazelcast Cluster Embedded Distributed Cache
- 23. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast.newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)
- 27. Application Application Load Balancer Cache Cache Request Hazelcast Cluster Embedded Distributed Cache
- 28. Embedded Cache Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications
- 29. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 30. 2. Client-Server
- 33. Application Load Balancer Application Request Cache Server Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team
- 38. Client-Server Cache $ ./start.sh Starting Hazelcast Cache Server (standalone)
- 39. Client-Server Cache Starting Hazelcast Cache Server (Kubernetes) $ helm install hazelcast/hazelcast
- 40. Client-Server Cache Hazelcast Client (Kubernetes): @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getKubernetesConfig() .setEnabled(true); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Starting Hazelcast Cache Server (Kubernetes) $ helm install hazelcast/hazelcast
- 41. Application Load Balancer Application Request Cache Server Client-Server Cache Ops Team Separate Management: ● backups ● (auto) scaling ● security
- 42. 2*. Cloud
- 43. Application Load Balancer Application Request Cloud (Cache as a Service)
- 44. Application Load Balancer Application Request Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team
- 45. Application Load Balancer Application Request Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team
- 46. Application Load Balancer Application Request Cloud (Cache as a Service)
- 47. @Configuration public class HazelcastCloudConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled(true) .setDiscoveryToken("KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setGroupConfig( new GroupConfig("test-cluster", "b2f984b5dd3314")); return new HazelcastCacheManager( HazelcastClient.newHazelcastClient(clientConfig)); } } Cloud (Cache as a Service)
- 49. Client-Server (Cloud) Cache Pros Cons ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)
- 50. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 51. 3. Sidecar
- 52. Kubernetes Service (Load Balancer) Request Hazelcast Cluster Kubernetes POD Application Container Cache Container Application Container Cache Container Kubernetes POD Sidecar Cache
- 53. Sidecar Cache Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses cache client to connect ● clear isolation between app and cache
- 54. @Configuration public class HazelcastSidecarConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress("localhost:5701"); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Sidecar Cache
- 55. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast
- 56. Sidecar Cache Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Limited to container-based environments ● Not flexible management (scaling, backup) ● Data collocated with application PODs
- 57. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 58. 4. Reverse Proxy
- 60. Reverse Proxy Cache http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }
- 61. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Cache Issues
- 62. NGINX Reverse Proxy Cache Issues ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk
- 63. 4*. Reverse Proxy Sidecar
- 64. Kubernetes Service (Load Balancer) Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Cache Container Application Container Reverse Proxy Cache Container Kubernetes POD Reverse Proxy Sidecar Cache
- 66. Good
- 67. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 68. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 69. Reverse Proxy Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers: - name: init-networking image: leszko/init-networking containers: - name: caching-proxy image: leszko/caching-proxy - name: application image: leszko/application
- 70. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 72. Reverse Proxy Sidecar Cache (Istio)
- 73. Bad
- 75. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:
- 76. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }
- 77. Reverse Proxy (Sidecar) Cache Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)
- 78. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda
- 79. Summary
- 84. application-aware? containers? Reverse ProxyReverse Proxy Sidecar lot of data? security restrictions? yes no yes no
- 85. application-aware? containers? Reverse ProxyReverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? yes no yes nono
- 86. application-aware? containers? Reverse ProxyReverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) yes no yes no no no
- 87. application-aware? containers? Reverse ProxyReverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar yes no yes yes no no no
- 88. application-aware? containers? Reverse ProxyReverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no
- 89. application-aware? containers? Reverse ProxyReverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no nono no
- 90. application-aware? containers? Reverse ProxyReverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-ServerCloud yes no yes yes yes yes no nono no
- 91. Resources ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Caching Prototype: https://github.com/leszko/caching-injector ● Caching Best Practices: https://vladmihalcea.com/caching-best-practices/ ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching /