Blackboard DevCon 2013 - Advanced Caching in Blackboard Learn Using Redis Building Block
Download as PPTX, PDF2 likes2,581 views
Redis Cache B2 is a caching framework add-on for Blackboard Learn that uses Redis as an additional memory store outside the Java heap. It provides better performance and availability compared to the default Ehcache implementation. When the database or JVM heap is under pressure, response times and transaction rates improve with Redis Cache B2. It also reduces cache invalidations during updates and allows the cache to survive application restarts. Redis Cache B2 offers easy management through monitoring tools and fails safely without performance degradation when Redis is unavailable. It was released in August 2013 for Blackboard Learn versions 9.1 SP11 and above.
Blackboard DevCon 2013 - Advanced Caching in Blackboard Learn Using Redis Building Block
- 1. Blackboard Learn Redis Cache B2 Noriaki Tatsumi, Director, Engineering Blackboard, Inc.
- 2. 2 Redis Cache B2 What is it? Enables Blackboard Learn’s caching framework to use Redis as an additional type of memory store outside the Java heap.
- 3. 3 Redis What is it? • Open source @ http://redis.io • Networked advanced key-value store • Powerful but simple + easy • Written in ANSI C • Companies that use Redis – Craigslist – Github – Stackoverflow – Instagram
- 4. 4 Caching in Learn Ehcache RMI Replicated Caching + Local Disk Overflow
- 5. 5 Redis Cache B2 Ehcache RMI Replicated Caching + Redis Overflow
- 6. 6 Performance Test Test case: Database is pressured Redis Cache B2 App Nodes %Fail Total Txns 95th% Resp Time (s) 90th% Resp Time (s) Stdev Disabled 11 0.40 235,852 14.2 10.61 6.09 Enabled 11 0.35 276,602 9.05 7.2 4.03 Delta 40,750 -5.15 -3.41 -2.06 Delta (%) 17.28% -36.27% -32.14% - 33.83% 0 5 10 15 95th% Resp Time(s) 90th% Resp Time(s) Ehcache Redis Cache B2 0 100,000 200,000 300,000 Ehcache Redis Cache B2 Total Txns
- 7. 7 Performance Test Test case: JVM heap is pressured Redis Cache B2 App Nodes %Fail Total Txns 95th% Resp Time (s) 90th% Resp Time (s) Stdev Disabled 11 0.61 292,864 5.86 4.41 18.49 Enabled 11 0.69 317,058 4.8 3.81 14.77 Delta 24,194 -1.06 -0.6 -3.72 Delta (%) 8.26% -18.09% -13.61% - 20.12% 0 2 4 6 8 95th% Resp Time(s) 90th% Resp Time(s) Ehcache Redis Cache B2 0 100,000 200,000 300,000 400,000 Ehcache Redis Cache B2 Total Txns
- 8. 8 Performance Test Test case: No pressure on any component Redis Cache B2App Nodes %Fail Total Txns 95th% Resp Time (s) 90th% Resp Time (s) Stdev Disabled 11 0.57 361,817 4.22 3.39 1.67 Enabled 11 0.47 363,220 4.02 3.28 1.69 Delta 1,403 -0.20 -0.11 +0.02 Delta (%) 0.39% -4.74% -3.24% -1.20% 0 1 2 3 4 5 95th% Resp Time(s) 90th% Resp Time(s) Ehcache Redis Cache B2 0 100,000 200,000 300,000 400,000 Ehcache Redis Cache B2 Total Txns
- 10. 10 Why Use Redis Cache B2? No risk of performance loss – Redis fails safe
- 11. 11 Why Use Redis Cache B2? Cache is shared and the hottest subset of data is kept local
- 12. 12 Why Use Redis Cache B2? Cache is available under JVM heap pressure PanicRoom thread • Triggers when free heap memory is less than 129MB • Flushes cache to avoid running out of memory • Log message: actuallyFreeMemory: Starting to free up registered caches actuallyFreeMemory: Finished freeing registered caches
- 13. 13 Why Use Redis Cache B2? Keeps a manageable JVM heap size From performanceterracotta.blogspot.com • GC pause increases with heap size • Compressed oops feature limited to 32 GB heap size
- 14. 14 Why Use Redis Cache B2? Bigger cache and faster overflow storage JVM Heap Network Memory Disk Size Speed
- 15. 15 Why Use Redis Cache B2? Fast and efficient object graph serialization Cache Java bytes Kryo bytes class blackboard.data.course.Course 4506 1837 class blackboard.data.navigation.CourseNavigationApplication 2551 621 class blackboard.data.user.User 7304 1735 class blackboard.platform.session.impl.BbSessionImpl 8107 2119 0 5000 10000 Java bytes Kryo bytes
- 16. 16 Why Use Redis Cache B2? Asynchronous overflow operations with write-behind strategy Test case: 10,000 serial iterations per operation X 5 runs Test Put: Mean(s) Put: Min(s) Put: Max(s) Put: StdDev(s) Disk overflow on SSD 3.116242472 3.053443009 3.315864087 0.11230668 Redis overflow 0.033371059 0.030807671 0.038541155 0.003137204 -3.082871413 -3.022635338 -3.277322932 -0.109169476 -98.93% -98.99% -98.84% -97.21% 0 1 2 3 4 Put: Mean(s) Put: Min(s) Put: Max(s) Put: StdDev(s) Disk overflow on SSD Redis overflow
- 17. 17 Why Use Redis Cache B2? Reduces on-heap cluster invalidation overhead - • On-heap cache size - • P(updates + removes) - • Cluster invalidations
- 18. 18 Why Use Redis Cache B2? Cache survives application restarts and crashes
- 19. 19 Monitoring and Management Rich monitoring features for easy management
- 20. 20 Monitoring and Management Rich monitoring features for easy management
- 21. 21 Monitoring and Management Rich monitoring features for easy management
- 22. 22 Monitoring and Management Rich monitoring features for easy management
- 23. 23 Monitoring and Management Rich monitoring features for easy management
- 24. 24 Summary of Redis B2 • Provides better user experience from performance and availability boost • Easy to manage – Redis is simple and easy to install/operate – Redis cache fails safe – Redis Cache B2 has tools to monitor and diagnose problems • When do you need it? – High cache evictions – Database pressure – JVM heap pressure
- 25. 25 Release • Date: August 2013 • Requirements: 9.1 SP 11+ – Cumulative Patch 8 for 9.1 SP 11 – Cumulative Patch 2 for 9.1 SP 12
- 26. 26 Learn More! help.blackboard.com • Performance, scalability, and availability highlights • Installation guide for Redis server and Redis B2 • Hardening Redis server security
- 27. 27 Thank You! Noriaki Tatsumi Director, Engineering Blackboard, Inc. noriaki.tatsumi@blackboard.com
Editor's Notes
- #2: Good morning. My name is Noriaki Tatsumi, Director of engineering. I’d like to introduce you to a new building block called Redis Cache B2 that’s scheduled to come out next month.
- #3: Redis Cache B2 is a low level functionality that extends Blackboard Learn’s caching framework. It essentially adds an additional type of memory outside the Java heap.
- #4: It leverages a popular open source NoSQL server used by prominent companies like craigslist, github… It’s often seen as a competitor to Memcached but it’s much simpler and easier to use and has many more advanced functionalities. Redis has an excellent documentation on their website at http://redis.io so I highly encourage checking it out.
- #5: Because it’s the key to scaling large environment. Services like Facebook, Twitter, Craigslist would not work without caching.
- #6: So I want to give you a visual representation of how Redis is placed in the deployment architecture. This is what caching in Learn looks like out of the box.Each application node has its own copy of cacheTalks to each other to keep in sync using peer discovery service to find each otherDoes not copy data, only invalidations when updates and removal of cache happens
- #7: When using Redis Cache B2. A centralized instance of Redis server is deployed for all application nodes. Before going into more details, I’d like to show you some performance benchmark results of using Redis.
- #8: This benchmark test mimicked the scenario when database is pressured by load. For all of our performance tests, we used 2 to 1 ratio for database data volume and the JVM heap cache size.
- #9: Here’s another test case for when JVM heap is pressured.
- #11: This is statistics from Wikia site. It illustrates that performance and adoption rate are directly linked! Users don’t tolerate slow apps. You want your students to be kept engaged.2 second latency can double the abandonment5 seconds can triple
- #12: Now, I’d like to discuss the advantages that Redis cache B2 brings in detail.
- #13: As you saw in the diagram, Redis is deployed as a centralized caching server. This enables Redis to share cache among application nodes and also keep he hottest data local.
- #14: Blackboard Learn has a safety mechanism called, PanicThread.
- #15: Redis keeps JVM heap size smaller.
- #16: Redis offers network memory, which gives speed advantage over disk overflow.
- #17: We use a library called Kryo to make sure the serialization is efficient as possible.
- #18: Compared to disk overflow implementation we have in Blackboard Learn, Redis is very efficient at writing cache. It uses write-behind strategy, which means its writes are processed asynchronously. Therefore, even on SSD, Redis cache write operation blows away disk overflow by huge number.
- #19: For a very large Learn deployment with over 10 application nodes, we’re seeing big overhead from cluster invalidation in the field. P = probability
- #25: High max saturation and evicted cache require Redis
- #26: High max saturation and evicted cache require Redis