The Path to ScyllaDB 5.2
0 likes560 views
The document outlines the enhancements in ScyllaDB 5.2, including improved streaming robustness, autoparallel queries, and WebAssembly user-defined functions. It introduces features like per-partition throttling, consistent schema and topology, and improved handling of tombstones to enhance resilience and performance. Additionally, it discusses the integration of new SSD disk modeling and tiered storage options for optimized cost and latency.
The Path to ScyllaDB 5.2
- 1. ScyllaDB 5.2 and Beyond Fresh from the ScyllaDB Oven Avi Kivity, CTO and Co-Founder
- 2. Agenda ■ Increasing Streaming Robustness ■ Autoparallel Queries ■ WebAssembly User Defined Functions ■ Per-partition Throttling ■ Alternator Updates ■ Consistent Schema and Topology ■ New SSD Disk Modeling ■ Taming Corner Cases ■ What’s Cooking Now
- 3. Repair-Based Node Operations ■ Resumable bootstrap/decommission ■ Stream from primary replica ■ Or a quorum if primary is missing ■ Increases resilience and improves correctness
- 4. Autoparallel Queries ■ Aggregations previously done via Spark or custom code ■ Instead, recognize certain CQL patterns ■ Dispatch to all nodes, all vcpus within nodes Node 5 Node 1 Node 2 Node 4 Node 3 SELECT COUNT(*) FROM t
- 5. WebAssembly UDF/UDA ■ Push compute into database ■ Use any language* ■ Computations run in a WASM sandbox ■ Use case: analytics *as long as it’s Rust
- 6. Per Partition Rate Limit ■ New CQL table attribute to limit access rate to partition ■ Works for reads and writes ■ Prevent bot accounts from spamming database ■ “Hot partition”
- 7. Alternator Updates ■ Time-to-live Expiration ■ Improved performance
- 8. ■ Eliminate classes of operator errors ■ Concurrent schema changes ■ Concurrent topology operations ■ Lay groundwork for more advanced features ■ Concurrent node bootstrap/decommission ■ Tablets ■ Strong consistency Consistent Schema and Topology
- 9. ScyllaDB knows more about the disk operating envelope New SSD Disk Modeling
- 11. Reverse Queries ■ 4.5 and older slow for large partitions ■ 4.6 fast, but skipped cache ■ 5.0+ fast, supports cache ■ Works well with paging SELECT * FROM tab WHERE … ORDER BY clustering_key DESC
- 12. ■ Queries that encounter large consecutive tombstone runs are now well supported ■ Partitions with many range tombstones work well Better Handling of Tombstones
- 13. ■ Escalating countermeasures as memory usage increases ■ Prevent new queries from starting ■ Allow only one query to make progress ■ Kill all but one query Improved Out-of-Memory Handling
- 14. Repair-Based Tombstone Garbage Collection ■ Eliminate gc_grace_seconds ■ Tie tombstone garbage collection to last repair ■ Improves performance for clusters that have frequent repair ■ Improves correctness for clusters that missed repair
- 15. Cooking Now
- 16. Nudging the CQL Grammar Towards SQL ■ Relaxing constraints ■ Reconciling semantic oddities ■ Increasing the scope of autoparallel queries
- 17. ■ A spectrum of cost/performance tradeoffs ■ RAM: Extremely fast (100ns), very expensive ■ NVMe: Very fast, (100µs), expensive ■ HDD: Slow (10ms), cheap ■ Cloud Object storage (S3 and similar) ■ Slow (40ms), cheap ■ Infinitely expandable ■ Easy to manipulate ■ Shared access Object Storage
- 18. ■ Very dense databases ■ Where latency is not critical ■ Tiered storage ■ Mix service levels and cost ■ Optimize both cost and latency Use Cases for Object Storage
- 19. Thank You Stay in Touch Avi Kivity avi@scylladb.com @AviKivity @avikivity