Architectural caching patterns for kubernetes
- 1. Architectural Caching Patterns for Kubernetes Rafał Leszko @RafalLeszko rafalleszko.com Hazelcast
- 2. About me ● Cloud Software Engineer at Hazelcast ● Worked at Google and CERN ● Author of the book "Continuous Delivery with Docker and Jenkins" ● Trainer and conference speaker ● Live in Kraków, Poland
- 3. About Hazelcast ● Distributed Company ● Open Source Software ● 140+ Employees ● Products: ○ Hazelcast IMDG ○ Hazelcast Jet ○ Hazelcast Cloud @Hazelcast www.hazelcast.com
- 4. ● Introduction ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 5. Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby Microservice World
- 6. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache cache
- 7. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
- 8. Microservice World Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby cache cache cache
- 9. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ○ Embedded Distributed ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 10. 1. Embedded
- 12. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 13. private ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); private String processRequest(String request) { if (cache.contains(request)) { return cache.get(request); } String response = process(request); cache.put(request, response); return response; } Embedded Cache (Pure Java implementation)
- 14. ● No Eviction Policies ● No Max Size Limit (OutOfMemoryError) ● No Statistics ● No built-in Cache Loaders ● No Expiration Time ● No Notification Mechanism Java Collection is not a Cache!
- 16. Embedded Cache (Java libraries) CacheBuilder.newBuilder() .initialCapacity(300) .expireAfterAccess(Duration.ofMinutes(10)) .maximumSize(1000) .build();
- 17. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } }
- 18. Caching Application Layer @Service public class BookService { @Cacheable("books") public String getBookNameByIsbn(String isbn) { return findBookInSlowSource(isbn); } } Be Careful, Spring uses ConcurrentHashMap by default!
- 21. Application Application Kubernetes Service Cache Cache Request Hazelcast Cluster Embedded Distributed Cache Kubernetes Pod Kubernetes Pod
- 22. @Configuration public class HazelcastConfiguration { @Bean CacheManager cacheManager() { return new HazelcastCacheManager( Hazelcast.newHazelcastInstance()); } } Embedded Distributed Cache (Spring with Hazelcast)
- 25. Embedded Distributed Cache Application Application Kubernetes Service Cache Cache Request Hazelcast Cluster Kubernetes Pod Kubernetes Pod
- 26. Embedded Cache Pros Cons ● Simple configuration / deployment ● Low-latency data access ● No separate Ops Team needed ● Not flexible management (scaling, backup) ● Limited to JVM-based applications ● Data collocated with applications
- 27. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ○ Cloud ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 28. 2. Client-Server
- 29. Application Kubernetes Service Application Request Cache Server StatefulSet Client-Server Cache Kubernetes Pod Kubernetes Pod
- 30. Client-Server Cache Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 31. Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 32. Client-Server Cache Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 33. Client-Server Cache Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 36. Client-Server Cache Starting Hazelcast Cache Server $ helm install hazelcast/hazelcast
- 37. Client-Server Cache Hazelcast Client @Configuration public class HazelcastClientConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getKubernetesConfig() .setEnabled(true); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Starting Hazelcast Cache Server $ helm install hazelcast/hazelcast
- 38. Client-Server Cache Separate Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Cache Server StatefulSet Kubernetes Pod Kubernetes Pod
- 39. 2*. Cloud
- 40. Application Kubernetes Service Application Request Cloud (Cache as a Service) Kubernetes Pod Kubernetes Pod
- 41. Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 42. Cloud (Cache as a Service) Management: ● backups ● (auto) scaling ● security Ops Team Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 43. Cloud (Cache as a Service) Application Kubernetes Service Application Request Kubernetes Pod Kubernetes Pod
- 44. @Configuration public class HazelcastCloudConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig().getCloudConfig() .setEnabled(true) .setDiscoveryToken("KSXFDTi5HXPJGR0wRAjLgKe45tvEEhd"); clientConfig.setGroupConfig( new GroupConfig("test-cluster", "b2f984b5dd3314")); return new HazelcastCacheManager( HazelcastClient.newHazelcastClient(clientConfig)); } } Cloud (Cache as a Service)
- 45. Client-Server (Cloud) Cache Pros ● Data separate from applications ● Separate management (scaling, backup) ● Programming-language agnostic Cons ● Separate Ops effort ● Higher latency ● Server network requires adjustment (same region, same VPC)
- 46. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 47. 3. Sidecar
- 48. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Cache Container Application Container Cache Container Kubernetes POD Sidecar Cache
- 49. Sidecar Cache Similar to Embedded: ● the same physical machine ● the same resource pool ● scales up and down together ● no discovery needed (always localhost) Similar to Client-Server: ● different programming language ● uses cache client to connect ● clear isolation between app and cache
- 50. @Configuration public class HazelcastSidecarConfiguration { @Bean CacheManager cacheManager() { ClientConfig clientConfig = new ClientConfig(); clientConfig.getNetworkConfig() .addAddress("localhost:5701"); return new HazelcastCacheManager(HazelcastClient .newHazelcastClient(clientConfig)); } } Sidecar Cache
- 51. Sidecar Cache apiVersion: apps/v1 kind: Deployment ... spec: template: spec: containers: - name: application image: leszko/application - name: hazelcast image: hazelcast/hazelcast
- 52. Sidecar Cache Pros Cons ● Simple configuration ● Programming-language agnostic ● Low latency ● Some isolation of data and applications ● Not flexible management (scaling, backup) ● Data collocated with application PODs
- 53. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ○ Reverse Proxy Sidecar ● Summary Agenda
- 54. 4. Reverse Proxy
- 55. Application Kubernetes Service Cache Application Request Reverse Proxy Cache Kubernetes Pod Kubernetes Pod
- 56. Reverse Proxy Cache http { ... proxy_cache_path /data/nginx/cache keys_zone=one:10m; ... }
- 57. ● Only for HTTP ● Not distributed ● No High Availability ● Data stored on the disk NGINX Reverse Proxy Cache Issues
- 58. 4*. Reverse Proxy Sidecar
- 59. Kubernetes Service Request Hazelcast Cluster Kubernetes POD Application Container Reverse Proxy Cache Container Application Container Reverse Proxy Cache Container Kubernetes POD Reverse Proxy Sidecar Cache
- 61. Good
- 62. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 63. Reverse Proxy Sidecar Cache Service 1 Service 2 v1 Service 2 v2 Service 1 Service 4 v1 Service 4 v2 Service 4 v3 Ruby
- 64. Bad
- 66. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache:
- 67. Reverse Proxy Cache @CacheEvict(value = "someValue", allEntries = true) public void evictAllCacheValues() {} Application Cache: Proxy Cache: http { ... location / { add_header Cache-Control public; expires 86400; etag on; } }
- 68. Reverse Proxy (Sidecar) Cache Pros Cons ● Configuration-based (no need to change applications) ● Programming-language agnostic ● Consistent with containers and microservice world ● Difficult cache invalidation ● No mature solutions yet ● Protocol-based (e.g. works only with HTTP)
- 69. ● Introduction ✔ ● Caching Architectural Patterns ○ Embedded ✔ ○ Embedded Distributed ✔ ○ Client-Server ✔ ○ Cloud ✔ ○ Sidecar ✔ ○ Reverse Proxy ✔ ○ Reverse Proxy Sidecar ✔ ● Summary Agenda
- 70. Summary
- 74. application-aware? Reverse Proxy Reverse Proxy Sidecar no yes no early adopter?
- 75. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? yes no yes no early adopter?
- 76. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? yes no yes nono early adopter?
- 77. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) yes no yes no no no early adopter?
- 78. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar yes no yes yes no no no early adopter?
- 79. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? yes no yes yes yes no no no early adopter?
- 80. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-Server yes no yes yes yes no nono no early adopter?
- 81. application-aware? Reverse Proxy Reverse Proxy Sidecar lot of data? security restrictions? language-agnostic? Embedded (Distributed) Sidecar cloud? Client-ServerCloud yes no yes yes yes yes no nono no early adopter?
- 82. Resources ● Hazelcast Sidecar Container Pattern: https://hazelcast.com/blog/hazelcast-sidecar-container-pattern/ ● Hazelcast Reverse Proxy Sidecar Caching Prototype: https://github.com/leszko/caching-injector ● Caching Best Practices: https://vladmihalcea.com/caching-best-practices/ ● NGINX HTTP Reverse Proxy Caching: https://www.nginx.com/resources/videos/best-practices-for-caching/