Intel and MariaDB: web-scale applications with distributed logs
2 likes355 views
The document outlines a strategic collaboration between MariaDB and Intel to enhance database functionality and performance through co-marketing, early access to hardware, and joint software development. It emphasizes the necessity for cloud-native, open-source databases that support big data applications with unlimited scalability and concurrency. Key features discussed include the integration of MyRocks with RocksDB and the use of Amazon RDS for deployment in cloud environments.
Intel and MariaDB: web-scale applications with distributed logs
- 1. Strategic collaboration to enhance MariaDB functionality, performance & TCO
- 2. 2
- 3. Intel Confidential – Internal use only 3 MariaDB* and MyRocks* storage engine Intel® Xeon® Scalable Platform WORKING TOGETHER TO OPTIMIZE THE COMBINATION OF: Co-Marketing Collaborations Intel® Optane™ SSDs Early access to Intel pre-production H/W GoToMarket support and customer Trusted Advisors Joint performance analysis and tuning S/W development tools Full tech. support Long-term development project collaboration MyRocks
- 4. David Cohen, Yekesa Kosuru, Steve Shaw, Mikhail Sinyavin, Dhruba Borthakur Jiten Vaidya, Sugu Sougoumarane Jan 24th 2019
- 5. 3 5 TRANSACTIONAL DATABASE WITH UNLIMITED SCALE AND CONCURRENCY • Enterprises with big data need unlimited scale, high concurrency, low Latency DISAGGREGATING STORAGE Storage and Compute must scale independently to accommodate non-linear data growth CLOUD-NATIVE, OPEN-SOURCE, AND AI/ANALYTICS- READY Cloud native database, open source, that can be readily integrated into their AI/analytics investments. 1 2
- 6. Queries Updates RocksDB Cloud Log Storage, Kafka* Topic Cloud Storage Bucket Flush sst file to cloud storage Persistent read cache on SSD Flush sst file to local SSD RocksDB-Cloud* writes reads WAL memtable cache block cache MariaDB (binlogs) MyRocks WALSST RocksDB local-attached storage bin bin bRocksDB-cloud Cloud Storage
- 7. • Scale-out • Vitess - Sharded cluster for scale • Each Shard is One Master + Multiple Slaves • Custom sharding key • Read Scale-out: add more slaves • Write Scale-out: add more shards • ACID compliance across cluster • Connection Pool, Restrain bad actors • Backups and recovery from cloud storage
- 8. 8 RocksDB-Cloud* Queries Updates Server Cloud Bucket A Cloud Log Storage, Kafka* write read read RocksDB-Cloud Queries Cloned Server Cloud Bucket B write read Clones are read-only replicas WAL writes WAL tailer
- 9. M 9 • Can use either InnoDB* or MyRocks* w/RocksDB* library • Requires Support for Lossless-SemiSync Protocol and GTIDs in the Slaves • Need a “Lambda” function to kickoff “Switchover” • Need Automation for carrying out Switchover • Uses MyRocks w/Rockset’s RocksDB-Cloud library • Requires Support for Lossless-SemiSync Protocol and GTIDs in the Log Tailer • Need a “Lambda” function to kickoff “Switchover” • Need Automation for carrying out Switchover 1. Master-Slave (Shared-Nothing) 2. Master-Only (Shared) Time0 Time1Time0 Time1 Vitess Vitess
- 10. http://db.cs.cmu.edu/papers/2017/p1753-arulraj.pdf LOGGING & RECOVERY DATA PLACEMENT ACCESS METHODS STORAGE MANAGER REPLICATION FILESYSTEM INTERFACE ACCESS INTERFERENCE Local Cache Cloud Native WAL Cloud Native Store Storage RocksDB
- 11. Intel Confidential – Internal use only 11 MariaDB* available as DBaS through Amazon Relational Database Service (RDS) – AWS* “ Intel instances offer both higher value and absolute performances across almost all cloud use cases ” Source: WCCFTECH First Look: Intel vs AMD EPYC AWS Cloud (IaaS) Benchmarks https://wccftech.com/first-look-intel-vs-amd-epyc-aws-cloud-iaas-benchmarks/ Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks.Intel does not control or audit third-party data. You should review this content, consult other sources, and confirm whether referenced data are accurate. Cost reduction scenarios described are intended as examples of how a given Intel- based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. .*Other names and brands may be claimed as property of others. Source: WCCFTECH First Look: Intel vs AMD EPYC AWS Cloud (IaaS) Benchmarks https://wccftech.com/first-look-intel-vs-amd-epyc-aws-cloud-iaas-benchmarks/