Introducing Scylla Open Source 4.0
1 like1,340 views
The document introduces Scylla Open Source 4.0, highlighting its new features such as a DynamoDB-compatible API (Alternator), lightweight transactions, change data capture, repair improvements, and a Kubernetes operator. Scylla 4.0 claims superior performance over Cassandra 4.0, showcasing significant advantages in throughput, latency, and cost-effectiveness. The presentation outlines the use cases, comparisons with other databases, and innovative enhancements designed to improve cloud database management.
![CQL Conditional Statement
> UPDATE employees SET join_date = '2018-05-19'
WHERE firstname = 'John' AND lastname = 'Doe'
IF join_date != null;
[applied]
-----------
False
9](https://image.slidesharecdn.com/introducingscylla4-200521215851/85/Introducing-Scylla-Open-Source-4-0-9-320.jpg)
Introducing Scylla Open Source 4.0
- 1. Dor Laor, ScyllaDB Co-Founder & CEO Avi Kivity, ScyllaDB Co-Founder & CTO Introducing Scylla Open Source 4.0
- 2. Dor Laor Dor Laor is the CEO of ScyllaDB. Previously, Dor was part of the founding team of the KVM hypervisor under Qumranet that was acquired by Red Hat. At Red Hat Dor was managing the KVM and Xen development for several years. Dor holds an MSc from the Technion and a Phd in snowboarding. Avi Kivity Avi Kivity, CTO of ScyllaDB, is known mostly for starting the Kernel-based Virtual Machine (KVM) project, the hypervisor underlying many production clouds. He has worked for Qumranet and Red Hat as KVM maintainer until December 2012. Avi is now CTO of ScyllaDB, a company that seeks to bring the same kind of innovation to the public cloud space. Presenters
- 3. Agenda ■ Alternator: DynamoDB-Compatible API ■ Lightweight Transactions ■ CDC ■ Repair Improvements ■ Scylla Operator for Kubernetes ■ Comparing Scylla 4.0 to Cassandra 4.0
- 4. Scylla Alternator: DynamoDB-Compatible API ■ Part of Scylla core, not a layer ■ Fits well to the Scylla model: Yet ‘another’ columnar database ■ Drop-in replacement for DynamoDB ■ Open source! ■ Consumable through {git, gcc, docker, k8s, rpm, DBaaS}
- 5. Scylla Alternator: Why Use? ■ Substantially better price/performance ■ Better tail latency ■ Easy development – docker & laptop ■ It’s OPEN ■ Multi-cloud, hybrid-cloud, your own cloud ■ Observability: Prometheus and Grafana ■ No limits on object sizes, partition sizes, etc. ■ No throttled requests ■ Workload consolidation ■ Workload prioritization (coming)
- 6. Scylla Alternator: When to Use? ■ Docker image – Easy & cheap development ■ Leave one datacenter on DynamoDB, others on ScyllaDB ■ Scylla Cloud – Easy managed database ■ Ultimately – Always!
- 7. Protocol Comparison: DynamoDB API vs CQL ■ We recently covered it in this blog post ■ HTTP native vs CQL ■ HTTP lack of request multiplexing ■ Textual vs. binary protocol ■ CQL prepared statement ■ CQL topology awareness ■ CRDT vs RMW, LWT ■ Schema-full vs. schema-less ■ Keyspaces vs. single table design
- 8. Lightweight Transactions – Linearizability ■ Long anticipated, finally, feature parity ;) ■ Based on Paxos implementation, like Cassandra ■ 3 round trips vs Cassandra’s 4 ■ More improvements
- 9. CQL Conditional Statement > UPDATE employees SET join_date = '2018-05-19' WHERE firstname = 'John' AND lastname = 'Doe' IF join_date != null; [applied] ----------- False 9
- 10. Conditional Batches BEGIN BATCH UPDATE tasks SET n_abandoned = 0 WHERE project_id = 1 IF n_abandoned > 0 DELETE FROM tasks WHERE project_id = 1 AND state = 'Abandoned' APPLY BATCH; 10
- 11. Lightweight Transactions ■ 3 round trips vs Cassandra’s 4 ■ Shard-aware driver ■ Safe/durable commitlog by default with the best performance ■ Good observability
- 12. Change Data Capture – CDC Consumable modification record for one or more tables in the database ■ Capture changes (write/update/delete) ■ Asynchronously readable by a consumer ■ Using CQL statements, protocol, and driver ■ Table level Granularity ■ Highly Available ■ Persistent ■ (Eventually) Consistent
- 13. Feeding Microservices Kafka CDC Stream Fraud Detection Data Lake Real Time Analysis Search
- 14. CREATE TABLE company.employees ( department text, last_name text, first_name text, age int, level int, PRIMARY KEY (department, last_name, first_name) ) WITH cdc = {/* CDC parameters go here */}; CREATE TABLE company.employees_scylla_cdc_log ( cdc$stream_id blob, cdc$time timeuuid, cdc$batch_seq_no int, cdc$operation tinyint, cdc$ttl bigint, department text, first_name text, last_name text, age int, cdc$deleted_age boolean, level int, cdc$deleted_level boolean, PRIMARY KEY (cdc$stream_id, cdc$time, cdc$batch_seq_no) ) CDC Log
- 15. CDC Write RF=3 CL = QUORUM (2)
- 16. Comparison Chart Cassandra DynamoDB MongoDB Scylla Consumer location on-node off-node off-node off-node Replication duplicated deduplicated deduplicated deduplicated Deltas yes no partial yes Pre-image no yes no optional Post-image no yes yes optional Slow consumer reaction Table stopped Consumer loses data Consumer loses data Consumer loses data Ordering no yes yes yes Filtering no no ? yes
- 17. CDC under the Hood ■ CDC tables use TWCS ● TWCS major compaction (unrelated to CDC) ■ CQL Bypass cache – per query bypass hint for standard CQL ■ New: CDC table shouldn’t be cached – Per table settings
- 18. Repair Improvements ■ Row level repair ● Minimizes data transfer ● Minimizes disk reads
- 19. Repair Improvements ■ Row level repair ■ Repair Based Node Operations ● Bootstrap, decommission, replace, removenode, rebuild ● Node replace is _safe_ ● Resumable in nature ● Efficient – rebuild will only bring the missing data
- 20. Repair Improvements ■ Row level repair ■ Repair Based Node Operations ■ Tune scheduling priority to accelerate bootstrap – Coming ■ Adjust way of adding repair sstables to main dataset – Coming ■ Network scheduling – Future ■ IBF – Invertible Bloom Filter – Future
- 21. Kubernetes Operator Easy to get started:
- 27. Cassandra 4.0 Features ■ Java 11, ZGC ■ Netty ■ Zero copy networking ■ Virtual Tables ■ Transient Replication (experimental) ■ Incremental Repair ■ Audit Logging ■ Pluggable Storage Engine
- 28. The Benchmark ■ Max IOPS while latency < 10ms P99 ■ 40 Cassandra i3.4xl ■ 4 Scylla i3.metal ■ Scylla delivers 600k ops at 12ms P99 ■ Cassandra delivers 200k ops at 85ms P99 ■ Scylla performs ● 3x throughput ● 7x better latency ● 2.5x cheaper deployment ● 10x fewer nodes ● 14x faster elastic scale out -> More cost ● 30x faster elastic scale in
- 29. I3.metal Cluster Guided Tour
- 30. Resources ■ Download Scylla Open Source ■ Scylla University ■ Test Drive ■ On-Demand Webinars ● Lightweight Transactions ● Change Data Capture (CDC) ● Scylla’s DynamoDB Compatible API We’ll soon email you these links and the recording
- 31. Q&A
- 32. Thank you