Caching 101: Caching on the JVM (and beyond)
1 like73 views
This document outlines a presentation on caching 101. The presentation covers caching theory, caching in Java, caching patterns, and scaling caches. It discusses concepts like cache hits, misses, and coherence. It presents Java caching APIs like JSR-107 and frameworks like Ehcache. Common caching patterns like cache-aside, cache-through, and write-behind are explained. The document also discusses scaling techniques for caches, such as moving data off-heap and clustering caches across multiple servers.
Caching 101: Caching on the JVM (and beyond)
- 1. @AurBroszniowski @ljacomet#Caching101 Caching 101: on the JVM and beyond Aurélien Broszniowski @AurBroszniowski Louis Jacomet @ljacomet Software AG / Terracotta ehcache.org
- 2. @AurBroszniowski @ljacomet#Caching101 Agenda • Caching theory • Caching in Java • Caching patterns • Scaling
- 3. @AurBroszniowski @ljacomet#Caching101 Who are we? • Aurélien Broszniowski • Works at Software AG / Terracotta since 2010 (startup days!) • Coding since the teenage days, will probably never stop • OSS developer, see for example the Rainfall performance testing framework • Louis Jacomet • Principal Software Engineer at Software AG / Terracotta since 2013 • Developper close to his forties who did not dodge all things management • Interests range from Concurrency to API design, with learning new things as the driving factor
- 4. @AurBroszniowski @ljacomet#Caching101 Who are you? • Who knows nothing about caching? • Who already uses caching in production? • Who had caching related problems in production? • Ehcache anyone?
- 5. @AurBroszniowski @ljacomet#Caching101 There are only two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton
- 7. @AurBroszniowski @ljacomet#Caching101 What is a cache? • Data structure holding a temporary copy of some data • Trade off between higher memory usage for reduced latency • Targets: • Data which is reused • Data which is expensive to compute or retrieve
- 8. @AurBroszniowski @ljacomet#Caching101 Locality of reference • Same values or related ones are frequently accessed • One form of predictable behaviour • heavily used in CPUs for branch prediction • Principle applied at different levels: CPUs, network and OS • Ends up being a desirable feature in applications • Application closer to hardware behaviour
- 9. @AurBroszniowski @ljacomet#Caching101 The long tail • Coined by Chris Anderson • Application of a Power Law probability distribution • Pareto Law or 80:20 rule
- 10. @AurBroszniowski @ljacomet#Caching101 Terminology • hit: when the cache returns a value • miss: when the cache does not have a value • cold: when the cache is empty • warm-up: phase during which the cache fills up • warm: when a cache is filled and helps the most
- 11. @AurBroszniowski @ljacomet#Caching101 Possible usages • CPU bound applications • Normal speedup through algorithm improvement or parallelisation • Cache can help by storing computation results • I/O bound applications • Normal speedup through disk or network upgrades • Cache can help by storing data locally
- 12. @AurBroszniowski @ljacomet#Caching101 Amdhal’s law • Parallelisation • P: parallel proportion • N: processor count
- 13. @AurBroszniowski @ljacomet#Caching101 Amdhal’s law • Sequential • p: times a part was sped up • f: fraction of time improved
- 14. @AurBroszniowski @ljacomet#Caching101 Cache coherence • P write A to X, P read X => returns A when no writes happened in between, must be valid for single processor too • P2 writes A to X, P1 reads X => returns A if “enough time” has elapsed • P1 writes A to X, P2 writes B to X => no one can ever see B then A at X
- 15. @AurBroszniowski @ljacomet#Caching101 Desired cache features • Capacity control / eviction • Data freshness / expiry • Data consistency / invalidation • Fault tolerant
- 17. @AurBroszniowski @ljacomet#Caching101 Google Guava And more … BlazingCache
- 18. @AurBroszniowski @ljacomet#Caching101 JSR-107 aka JCache aka javax.cache • JSR submitted in 2001 • Early draft in 2012 co-lead by Coherence and Ehcache representatives • Specification finalised in March 2014 • Specifies API and semantics for temporary, in-memory caching of Java objects, including object creation, shared access, spooling, invalidation, and consistency across JVM's
- 19. @AurBroszniowski @ljacomet#Caching101 javax.cache features • CacheManager managing … Cache! • Expiry • Integration (CacheLoader and CacheWriter) • Cache listeners • Entry processors • Annotations • Statistics and MBeans
- 20. @AurBroszniowski @ljacomet#Caching101 javax.cache API CacheManager repository, identified by URI and ClassLoader Named Cache repository, handles their lifecycle ConcurrentMap<K, V> sibling, major interaction point for your logic
- 21. @AurBroszniowski @ljacomet#Caching101 javax.cache integration • Spring caching abstraction since version 4.1 • Hibernate 5.2 has JCache integration for 2nd level cache • Does not include transactional support • More to come???
- 22. @AurBroszniowski @ljacomet#Caching101 Caching guidelines • Immutable keys • Favour immutable values • store by-value or by-ref semantics • Understand what you cache • Think hibernate entities with lazy attributes • Always account for a miss due to expiry or eviction
- 24. @AurBroszniowski @ljacomet#Caching101 Cache aside (miss) Application Cache Database
- 25. @AurBroszniowski @ljacomet#Caching101 Cache aside (hit) Application Cache Database
- 27. @AurBroszniowski @ljacomet#Caching101 Hibernate 2nd level cache • Stores dehydrated entities • 4 strategies • ReadOnly • NonStrictReadWrite • ReadWrite • Transactional
- 28. @YourTwitterHandle#DVXFR14{session hashtag} @AurBroszniowski @ljacomet#Caching101 Sum m ary
- 29. @AurBroszniowski @ljacomet#Caching101 Cache aside conclusions • Solution most seen out there • Spring, Play, Grails, … • Most often based on annotations • Tricky to get the concurrency and / or atomicity right • Especially when rolling you own • Suffers from thundering herd: multiple invocations when cache is cold
- 30. @AurBroszniowski @ljacomet#Caching101 Cache through (miss) Application Cache Database
- 31. @AurBroszniowski @ljacomet#Caching101 Cache through (hit) Application Cache Database
- 32. @AurBroszniowski @ljacomet#Caching101 Cache through (write) Application Cache Database
- 34. @YourTwitterHandle#DVXFR14{session hashtag} @AurBroszniowski @ljacomet#Caching101 Sum m ary
- 35. @AurBroszniowski @ljacomet#Caching101 Cache through conclusions • Requires different abstraction / modelling • Viewing the system of record through the cache may not be easy • Provides better guarantees and consistency as invalidation is no longer required • Outside of external system of record modifications • Cost of writing is paid by application thread putting in the cache
- 38. @YourTwitterHandle#DVXFR14{session hashtag} @AurBroszniowski @ljacomet#Caching101 Sum m ary
- 39. @AurBroszniowski @ljacomet#Caching101 Write behind conclusions • Impacts your domain modelling • Scales out your writes • Can benefit from batching and coalescing • Durability function of the queue persistence • Idempotent operations are important in a distributed setup where failures do happen
- 41. @AurBroszniowski @ljacomet#Caching101 Moving off heap • JVMs have issues with large heaps due to garbage collection • off heap is a nice alternative for data having a well known lifecycle • Perfect match for cache data • Performance hit due to required serialization in place of object reference
- 42. @AurBroszniowski @ljacomet#Caching101 Clustering • Cache shared between multiple machines • Different topologies • peer to peer • client - server • with or without client caching • Consistency becomes a much harder problem
- 43. @AurBroszniowski @ljacomet#Caching101 Probabilistically Bounded Staleness • How long for a write to be readable by all? • How many old versions are still around? • Depending on • network delay • node processing time • .. delayed replication (i.e. batching)
- 45. @AurBroszniowski @ljacomet#Caching101 Terracotta consistency: strong Cache Terracotta client Terracotta server Terracotta client Terracotta client
- 46. @AurBroszniowski @ljacomet#Caching101 Terracotta consistency: eventual Cache Terracotta client Terracotta server Terracotta client Terracotta client
- 47. @YourTwitterHandle#DVXFR14{session hashtag} @AurBroszniowski @ljacomet#Caching101 Sum m ary
- 48. @AurBroszniowski @ljacomet#Caching101 Conclusion • You know it all now! • Understand your needs and what caching could bring • Consider it early on • Patterns and topologies can have an impact
- 49. @YourTwitterHandle#DVXFR14{session hashtag} @AurBroszniowski @ljacomet#Caching101 Q & A