Caching methodology and strategies
- 1. Caching – Methodology & Strategies By Vu Van Tiep Manager, Head Of Web Platform - VNG Skype: tiepvv
- 2. Scale Up vs Scale Out Scale UP Scale Out
- 4. Agenda • What • Cache Layer • Cache Application/Service • Methodology • Strategies • Performance? • Demo • Q&A
- 5. Cache – What? • Is a hardware or software component that stores data so future requests for that data can be served faster; • The data stored in a cache might be the result of an earlier computation, or the duplicate of data stored elsewhere https://en.wikipedia.org/wiki/Cache_(computing) Application Cache Database Hit MissUpdate Cache
- 6. Cache Layer Client (Browser) Application Caching Database - Browser - Proxy - CDN - in-memory - Server Caching - Distributed in memory key/value - Service: • Memcache • Membase • Redis • … - Database caching • Entity In memory - DB: • MySQL • MongoDB • Casandra, LevelDB • … 1 2 3 4 5 6
- 8. Browser & Proxy Client (Browser) Web Server 1 2 Client (Browser) Client (Browser) Web ServerProx y Client (Browser) ProxyCache Cache 1 2 1 2 34 Response Header • Expires • Last-Modified • Etag • Cache-Control • Pragram
- 9. Browser Caching – Http Header
- 10. Browser Caching – Http Header
- 11. Browser Caching – Local Storage
- 13. Types/Levels of Cache Level 1 (L1) Cache It is also called primary or internal cache. It is built directly into the processor chip. It has small capacity from 8 Km to 128 Kb. Level 2 (L2) Cache It is slower than L1 cache. Its storage capacity is more, i-e. From 64 Kb to 16 MB. The current processors contain advanced transfer cache on processor chip that is a type of L2 cache. The common size of this cache is from 512 kb to 8 Mb. Level 3 (L3) Cache This cache is separate from processor chip on the motherboard. It exists on the computer that uses L2 advanced transfer cache. It is slower than L1 and L2 cache. The personal computer often has up to 8 MB of L3 cache.
- 14. L1 cache reference 0.5 ns Branch mispredict 5 ns L2 cache reference 7 ns Mutex lock/unlock 100 ns Main memory reference 100 ns Compress 1K bytes with Zippy 10,000 ns Send 2K bytes over 1 Gbps network 20,000 ns (1) Read 1 MB sequentially from memory 250,000 ns (2) Round trip within same datacenter 500,000 ns Disk seek 10,000,000 ns 1 + 2 < (3) Read 1 MB sequentially from network 10,000,000 ns Read 1 MB sequentially from disk 30,000,000 ns Send packet CA->Netherlands->CA 150,000,000 ns From Jeff Dean - google.com Latency Numbers Note: 1 ns = 10^-9 seconds
- 16. Serialize Disk / Network Java Object PHP Object C# Object Write ReadSerialize DeSerialize
- 17. Performance Comparison Text Serialize Json – size: 167 bytes, se: 6ms, de: 3ms XML – 235 bytes Binary Serialize Thrift – size: 64 bytes, se: 53ms, de:1ms ProtoBuf – size: 31 bytes, se: 34ms, de: 4ms Binary serialize take less space XML, Json ProtoBuf, Thrift Message Pack Apache Arvo … Framework
- 18. Compress/Decompress Name Ratio C.speed D.speed MB/s MB/s zstd 0.8.2 -1 2.877 330 940 zlib 1.2.8 -1 2.73 95 360 brotli 0.4 -0 2.708 320 375 QuickLZ 1.5 2.237 510 605 LZO 2.09 2.106 610 870 LZ4 r131 2.101 620 3100 Snappy 1.1.3 2.091 480 1600 LZF 3.6 2.077 375 790 As a reference, several fast compression algorithms were tested and compared on a Core i7-3930K CPU @ 4.5GHz, using lzbench, an open-source in-memory benchmark by @inikep compiled with GCC 5.4.0, with the Silesia compression corpus. https://github.com/facebook/zstd
- 20. Cache Strategies 1. Optimize Data Size 2. Hot Object 3. In-Process 4. Lazy load 5. Distributed
- 21. Optimize Data Size Name Ratio C.speed D.speed MB/s MB/s zstd 0.8.2 -1 2.877 330 940 zlib 1.2.8 -1 2.73 95 360 brotli 0.4 -0 2.708 320 375 QuickLZ 1.5 2.237 510 605 LZO 2.09 2.106 610 870 LZ4 r131 2.101 620 3100 Snappy 1.1.3 2.091 480 1600 LZF 3.6 2.077 375 790 CompressSerialize Method Size (smaller is better) Thrift — TCompactProtocol 278 (not bad) Thrift — TBinaryProtocol 460 Protocol Buffers 250 (winner!) RMI 905 REST — JSON 559 REST — XML 836 High Level Goals: • Transparent between multiple programming languages (PHP/Java/Cpp/…) • Maintain Right balance between: • Efficiency (how much time/space?) • Availability of existing libraries
- 22. Hot Object => Eviction Algorithms 2G Ram 10G Data ServerData How to cache?
- 23. Eviction Algorithms: Least Frequently Used (LFU) A B C D E F 7 6 3 2 4 1 A B C D E F 7 6 3 3 4 1 Access Most Frequently Used Least Frequently Used
- 24. Eviction Algorithms: Least Recently Used (LRU) 1 2 3 4 5 6 4 1 2 3 5 6 Access
- 27. Distributed Cache Server 1 Server 3 Server 2Server 4 Distributed Cache Client 1 Client 2 Client 3
- 28. Performance: Profiler is the key
- 29. Demo
- 30. Contact info: • Name: Vu Van Tiep • Email: tiepvv@vng.com.vn / vuvantiep@gmail.com • Skype: tiepvv Q&A Thank You!
- 31. Reference • https://en.wikipedia.org/wiki/Cache_(computing) • http://www.slideshare.net/IgorAnishchenko/pb-vs-thrift-vs-avro • https://github.com/facebook/zstd • http://www.codeproject.com/Articles/56138/Consistent-hashing