Making the case for write-optimized database algorithms / Mark Callaghan (Facebook)
4 likes740 views
This document details the advantages of write-optimized database algorithms, particularly focusing on the performance of RocksDB and its variants (MyRocks and MongoRocks) compared to traditional B-tree based systems. It emphasizes the importance of balancing read, write, and space efficiency in database operations and presents performance benchmarks demonstrating the higher throughput and lower space requirements of RocksDB. Additionally, it discusses the adaptability of algorithms to meet varying efficiency goals and highlights ongoing developments in open-source database technologies.
Making the case for write-optimized database algorithms / Mark Callaghan (Facebook)
- 1. Making the case for write-optimized database algorithms Mark Callaghan Member of Technical Staff, Facebook
- 2. RocksDB • Embedded key-value storage engine MyRocks • RocksDB storage engine for MySQL • coming to Percona Server and MariaDB Server MongoRocks • RocksDB storage engine for MongoDB • In Percona Server today RocksDB, MyRocks & MongoRocks
- 3. • Good read efficiency • Better write efficiency • Best space efficiency RocksDB, MyRocks & MongoRocks • Use less SSD • Use lower endurance SSD In some cases, better read efficiency is possible Efficient performance is the goal
- 4. Benchmarketing? Sysbench read-write, in-memory 0 50000 100000 150000 200000 1 2 4 8 16 24 32 40 48 64 80 96 128 MyRocks InnoDB
- 5. Better is not just about throughput tpmC iostat rKB/t iostat wKB/t vmstat CPU/t Size (GB) p99 response (ms) MyRocks +zlib 95680 2.19 2.02 528 82 29.9 InnoDB 91981 2.67 7.49 400 222 18.4 tpcc-mysql, 1000 warehouses, IO-bound • QPS - average throughput • QoS - worst case throughput • Efficiency - hardware per query
- 6. Peak performance is overrated Performance Performance & Efficiency Meet performance goals Then optimize for efficiency
- 7. RUM or RWS? • RUM = Read, Update, Memory • RWS = Read, Write, Space Efficiency • Synonym for amplification • Related to, but not equivalent to, performance. An algorithm can’t be optimal for all of read, write & space amplification • See Designing Access Methods: The RUM Conjecture • daslab.seas.harvard.edu/rum-conjecture RUM Conjecture
- 8. For any useful database algorithm there exists another useful algorithm that has better read, write or space amplification. Define: optimal
- 9. Read, Write • physical work per logical request Space • sizeof(database files) / sizeof(data) Define: amplification
- 10. Storage • random operations • bytes read • bytes written Define: work CPU • blocks (un)compressed • memory/cache operations • hash searches • key comparisons • CPU seconds
- 11. Basic operations: • point read • range read • put • delete Complex operations: • query • transaction Define: operation Update is one of: • put • point read, put • point read, delete, put
- 12. Everything is relative tpmC iostat rKB/t iostat wKB/t vmstat CPU/t Size (GB) MyRocks+zlib 95680 2.19 2.02 528 82 InnoDB 91981 2.67 7.49 400 222 InnoDB/ MyRocks 0.96 1.22 3.71 0.76 2.71 tpcc-mysql, 1000 warehouses, IO-bound
- 13. Efficiency: B-Tree Example: InnoDB Read • logN key compares Write • sizeof(page) / sizeof(row) Space • 1.5X if leaf pages are 2/3 full Update-in-Place B-Tree • InnoDB Copy-on-Write B-Tree • WiredTiger • LMDB
- 14. Efficiency: leveled LSM Example: RocksDB, Cassandra Read • logN + log(N/10) + log(N/100) + log(N/1000) key compares • point reads can use bloom filter Write • rewrite previously written rows • worse than size-tiered LSM, better than B-Tree Space • 1.1X
- 15. Efficiency: size-tiered LSM Example: RocksDB, Cassandra, HBase Read • more than leveled LSM • point reads can use bloom filter Write • rewrite previously written rows • better than leveled LSM, better than B-Tree Space • ~2X, worse than leveled LSM
- 16. Efficiency: summary Read Write Space B-Tree best good leveled LSM good for point good best size-tiered LSM best
- 17. Theory meets practice Access distribution • LSM benefits from skew Cache • B-Tree - prefer to have index in cache • LSM - prefer to have all but largest level in cache IO costs are hard to predict
- 18. Linkbench, IO-bound TPS iostat r/t iostat wKB/t CPU usecs/t Size (GB) p99 update (ms) MyRocks+zlib 28965 1.03 1.25 999 374 1 InnoDB 21474 1.16 19.70 914 14xx 6 InnoDB+zlib 20734 1.07 14.59 1199 880 6 MyRocks: best throughput & QoS, most efficient
- 19. Space efficiency • Fragmentation • Fixed page size • More per-row metadata • No prefix encoding (InnoDB) Why did RocksDB beat a B-Tree? Write efficiency • Uses more space = more data to write • Working set larger than cache • sizeof(page) / sizeof(row) • Double write buffer (InnoDB)
- 20. Page size & write amplification Page size TPS iostat wKB/t MyRocks+zlib 16kb 28965 1.25 InnoDB 4kb 24845 6.13 InnoDB 8kb 24352 10.52 InnoDB 16kb 21414 19.70
- 21. Advantage B-Tree • Fewer key comparisons • Less IO for range queries Read efficiency: B-Tree vs LSM? Advantage LSM • Uses less space = more data in cache • Prefix key encoding when uncompressed • Efficient writes saves IO for reads • Read-free index maintenance • Bloom filter Performance is complex
- 22. Save on writes, spend more on reads MyRocks zlib TPS InnoDB TPS Ratio (MyRocks / InnoDB) Disk array 2195 414 5.3 Slow SSD 23484 10143 2.3 Fast SSD 28965 21414 1.4
- 23. Read versus Write efficiency • Indexes - more & wider Read versus Space efficiency • Bloom filters • Compression • Indexes Write versus Space efficiency • RocksDB fanout • Size-tiered vs leveled • GC & defragmentation frequency Trading between R, W and S efficiency
- 24. Write versus space amplification Space vs write amplification for a log-based algorithm WriteAmplification 0 2.5 5 7.5 10 Space Amplification 1.11 1.25 1.33 1.67 2 • space amplification = 100 / %full • write amplification = 100 / (100 - %full)
- 25. One size doesn’t fit all • B-Tree + LSM sharing one redo log Adaptive algorithms • DBA sets high-level goals • Algorithm adapts to achieve them More open source • MyRocks in MariaDB Server & Percona Server • MongoRocks in Percona Server • More features in RocksDB What comes next? More performance results are coming • YCSB • sysbench • time series • bulk load • tpcc-mysql