Where is my cache? Architectural patterns for caching microservices by example
- 1. Where is my cache? Architectural patterns for caching microservices by example Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast
- 2. About me ● Cloud Software Engineer at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
- 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com
- 4. ● Introduction ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 5. Why Caching? ● Performance ○ Decrease latency ○ Reduce load ● Resilience ○ High availability ○ Lower downtime
- 6. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World
- 7. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache cache
- 8. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
- 9. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
- 10. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 11. 1. Embedded
- 13. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 14. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 15. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!
- 17. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();
- 18. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }
- 19. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!
- 22. Application Application Load Balancer Cache Cache Request Hazelcast Cluster Embedded Distributed Cache
- 23. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast.newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)
- 24. DEMO
- 28. Application Application Load Balancer Cache Cache Request Hazelcast Cluster Embedded Distributed Cache
- 29. Embedded Cache Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications
- 30. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 31. 2. Client-Server
- 34. Application Load Balancer Application Request Cache Server Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team
- 39. Client-Server Cache $ ./start.sh Starting Hazelcast Cache Server (standalone)
- 40. Client-Server Cache Starting Hazelcast Cache Server (Kubernetes) $ helm install hazelcast/hazelcast
- 41. Client-Server Cache Hazelcast Client (Kubernetes): @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getKubernetesConfig() .setEnabled(true); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Starting Hazelcast Cache Server (Kubernetes) $ helm install hazelcast/hazelcast
- 42. Application Load Balancer Application Request Cache Server Client-Server Cache Ops Team Separate Management: ● backups ● (auto) scaling ● security
- 43. 2*. Cloud
- 44. Application Load Balancer Application Request Cloud (Cache as a Service)
- 45. Application Load Balancer Application Request Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team
- 46. Application Load Balancer Application Request Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team
- 47. Application Load Balancer Application Request Cloud (Cache as a Service)
- 48. @Configuration public class HazelcastCloudConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled(true) .setDiscoveryToken("KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setGroupConfig( new GroupConfig("test-cluster", "b2f984b5dd3314")); return new HazelcastCacheManager( HazelcastClient.newHazelcastClient(clientConfig)); } } Cloud (Cache as a Service)
- 50. Client-Server (Cloud) Cache Pros ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic Cons ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)
- 51. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 52. 3. Sidecar
- 53. Kubernetes Service (Load Balancer) Request Hazelcast Cluster Kubernetes POD Application Container Cache Container Application Container Cache Container Kubernetes POD Sidecar Cache
- 54. Sidecar Cache Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses cache client to connect ● clear isolation between app and cache
- 55. @Configuration public class HazelcastSidecarConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress("localhost:5701"); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Sidecar Cache
- 56. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast
- 57. Sidecar Cache Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Limited to container-based environments ● Not flexible management (scaling, backup) ● Data collocated with application PODs
- 58. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 59. 4. Reverse Proxy
- 61. Reverse Proxy Cache http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }
- 62. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Cache Issues
- 63. NGINX Reverse Proxy Cache Issues ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk
- 64. 4*. Reverse Proxy Sidecar
- 65. Kubernetes Service (Load Balancer) Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Cache Container Application Container Reverse Proxy Cache Container Kubernetes POD Reverse Proxy Sidecar Cache
- 67. Good
- 68. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 69. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 70. Reverse Proxy Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: initContainers: - name: init-networking image: leszko/init-networking containers: - name: caching-proxy image: leszko/caching-proxy - name: application image: leszko/application
- 71. Reverse Proxy Cache Container Application Container eth0 :80 Reverse Proxy Sidecar Cache :80 Pod :8000 lo :80
- 72. Reverse Proxy Sidecar Cache #!/bin/bash # Forward TCP traffic on port 80 to port 8000 on the eth0 interface. iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j REDIRECT --to-port 8000
- 73. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 75. Reverse Proxy Sidecar Cache (Istio)
- 76. Reverse Proxy Sidecar Cache (Istio)
- 77. Bad
- 79. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:
- 80. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }
- 81. Reverse Proxy (Sidecar) Cache Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)
- 82. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda
- 83. Summary
- 88. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no
- 89. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? yes no yes nono
- 90. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) yes no yes no no no
- 91. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar yes no yes yes no no no
- 92. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no
- 93. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no nono no
- 94. application-aware? containers? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? containers? Embedded (Distributed) Sidecar cloud? Client-ServerCloud yes no yes yes yes yes no nono no
- 95. Resources ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Caching Prototype: https://github.com/leszko/caching-injector ● Caching Best Practices: https://vladmihalcea.com/caching-best-practices/ ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching/