The New MariaDB Offering: MariaDB 10, MaxScale and More
Download as PPTX, PDF4 likes7,072 views
The document details the new MariaDB 10 and its significant features, including its improved scalability, replication, performance, and security as a MySQL fork. It highlights community contributions, various storage engines, and ongoing development projects in MariaDB. The document also outlines the roadmap for future versions and enhancements planned for MariaDB.
The New MariaDB Offering: MariaDB 10, MaxScale and More
- 1. © SkySQL Corporation Ab. Company Confidential. The new MariaDB offering MariaDB 10, MaxScale, and more Serge Frezefond, Cloud Solution Architect, SkySQL 03/07/2014 1
- 2. © SkySQL Corporation Ab. Company Confidential. MariaDB intro 03/07/2014 2
- 3. © SkySQL Corporation Ab. Company Confidential. What is MariaDB? • A free fork of MySQL with extra features • features, requested by the users • Backward compatible • file formats, replication, configuration files • aim at 100% drop-in replacement • Community developed • 50% of maria-captains are from the community • no hidden agenda
- 4. © SkySQL Corporation Ab. Company Confidential. Who’s using it?
- 5. © SkySQL Corporation Ab. Company Confidential. Where to get it • Distributions: • RedHat Enterprise Linux (default in RHEL 7), Debian, Ubuntu (since 14.04), Fedora, Mageia (default), openSUSE (default), Gentoo, Slackware (default), Arch (default), ALTLinux, TurboLinux, Chakra Project (default), Kdu, … • FreeBSD, OpenBSD • Mac OS X with MacPorts or Homebrew • From MariaDB.org • sources, binaries in .tar.gz or .zip (Windows) • Windows MSI installer • MariaDB apt and yum repositories
- 6. © SkySQL Corporation Ab. Company Confidential. Before MariaDB 10.0 • MariaDB 5.1 – 17 releases since Oct 2009 • Aria, XtraDB, PBXT, FederatedX • table elimination, pool of threads, • MariaDB 5.2 – 15 releases since Apr 2010 • OQGraph, SphinxSE • virtual columns, pluggable auth, segmented key cache, extended statistics, • MariaDB 5.3 – 13 releases since July 2011 • major optimizer improvements • replication: group commit, checksums • HandlerSocket, dynamic columns, Windows performance improvements, microsecond support, GIS precise operations, progress reporting, …
- 7. © SkySQL Corporation Ab. Company Confidential. Before MariaDB 10.0 • MariaDB 5.5 – 20 releases since Feb 2012 • new thread pool • non-blocking client API • extended keys optimization • dynamic replicate_do_*, replicate_ignore_*, replicate_wild_* variables
- 8. © SkySQL Corporation Ab. Company Confidential. Other MariaDB projects • MaxScale • MariaDB Galera Cluster • Connectors/Drivers • MariaDB C Client Library (LGPL) • MariaDB JDBC driver (LGPL) • MariaDB ODBC driver (LGPL) • MariaDB Audit Plugin
- 9. © SkySQL Corporation Ab. Company Confidential. Community activity • Past month (March 2014): • Added/Changed Articles in the KB: 123 • On Freenode IRC #maria, 550 people wrote 10173 lines • Source code (Launchpad): • 25 active branches • 395 commits 03/07/2014 9
- 10. © SkySQL Corporation Ab. Company Confidential. Community Contributions MariaDB 10.0 major contributions: • Per thread memory counting and usage • Base code and idea by Lixun Peng, Taobao • Multi-source replication • Base code by Lixun Peng, Taobao • GET_LOCK • Code by Konstantin "Kostja" Osipov, mail.ru • CONNECT storage engine • Code by Olivier Bertrand • Spider storage engine metadata_lock_info Information schema • Code by Kentoku Shiba, Spiral Arms • Roles • Code by Vicentiu Ciorbaru, Google Summer of Code 2013 • PCRE Regular Expressions • Code by Sudheera Palihakkara, Google Summer of Code 2013 • Global Transaction IDs • Some patches by Pavel Ivanov, Google 03/07/2014 10 https://mariadb.com/kb/en/log-of-mariadb-contributions/
- 11. © SkySQL Corporation Ab. Company Confidential. Community Ongoing Dev Projects • WebScaleSQL patches https://mariadb.atlassian.net/browse/MDEV-6039 • LevelDB/RocksDB storage engine (persistent key-value store for fast storage) https://mariadb.com/kb/en/leveldb-storage-engine/ • Fusion-IO page compression https://blog.mariadb.org/significant-performance- boost-with-new-mariadb-page-compression-on- fusionio/ • Kerberos authentication plugin https://mariadb.atlassian.net/browse/MDEV-4691 • Mroonga storage engine (fast fulltext search) https://mariadb.atlassian.net/browse/MDEV-5222 • Statistically optimize mysql-test runs by running less tests, Pablo Estrada, Google Summer of Code https://mariadb.atlassian.net/browse/MD EV-5776 • CREATE OR REPLACE, CREATE IF NOT EXISTS, DROP IF EXISTS, Sriram Patil, Google Summer of Code https://mariadb.atlassian.net/browse/MD EV-5359 • Self-Tuning Optimizer, Anshu Avinash, Google Summer of Code https://mariadb.atlassian.net/browse/MD EV-350 • Support of UseServerPrepStmts to MariaDB Java Connector, Puneet Dewan, Google Summer of Code 03/07/2014 11
- 12. © SkySQL Corporation Ab. Company Confidential. MariaDB 10.0
- 13. © SkySQL Ab. Company Confidential. MariaDB 10 ● MariaDB 10.0 GA launched March 31st, 2014. ● Application-compatible with MySQL. ● Includes significant unique new features in scalability, replication, performance, NoSQL, operations, security. ● Available under the GPL v2.
- 14. © SkySQL Ab. Company Confidential. MariaDB 10 Scalability: Parallel Slave Replication ● Sponsored by Google. ● Allows slaves to process update events in parallel. ● Uses MariaDB 10’s improved Global Transaction ID (GTID). ● MariaDB unlike MySQL can process multiple updates for a single database in parallel. Much better throughput. ● Preliminary benchmarks: almost 10x faster at 12 threads. Improves scalability, reduces slave lag, making slaves more consistent. MySQL MariaDB ✘ Single threaded per database. ✔ in 10.0.
- 15. © SkySQL Ab. Company Confidential. Parallel Slave Benchmark sysbench OLTP single database slave tps relative to master
- 16. © SkySQL Ab. Company Confidential. MariaDB 10 Scalability: Multi-Source Replication ● Collects data for analytics using built-in replication. ● Aids in administration example: consolidated backups of multiple databases. ● Uses MariaDB 10’s improved Global Transaction ID (GTID). Easier analytics, more insight, simpler administration, fewer headaches. Online E- Commerce Application Master S S S S Content Management System Click-stream data Data WarehouseSlave ETL Master S S S S Master S S S S MySQL MariaDB ✘ ✔ in 10.0.
- 17. © SkySQL Ab. Company Confidential. MariaDB 10 Scalability: Sharding with Spider ● Contributed by a 3rd party developer. ● Storage engine partitions large tables across multiple DBs. ● No changes to the client application. ● Developers aren’t aware, can keep applications simple. ● Allows for more parallelism, scale-out. ● Transactional storage engine. Web-scale, without the development hassle. Application Customers A-H I-P Q-Z Spider Customers A-H Shard 1 Customers I-P Shard 2 Customers Q-Z Shard 3 MySQL MariaDB ✘ optional 3rd party install ✔ in 10.0. Technology Preview
- 18. © SkySQL Ab. Company Confidential. MariaDB 10 Performance: TokuDB Storage Engine ● 3rd party partner - TokuTek. ● Drop-in replacement for InnoDB/XtraDB. ● Advanced indexing and compression algorithms. ● Transactional. ● Up to 20x performance gain for inserts/updates. ● Up to 90% less disk storage. ● Online schema changes. ● Reduces or eliminates slave lag. ● Best when DB doesn’t fit in memory, and SSD longevity is important. MySQL MariaDB ✘ optional 3rd party install ✔ in 10.0.
- 19. © SkySQL Ab. Company Confidential. MariaDB 10 Performance: Optimizer Improvements ● Of 29 distinct enhancements noted, 28 are in MariaDB 10. Just 1 only in MySQL 5.6. ● Enhancements include: ○ Disk access optimizations. ○ JOIN optimizations. ○ Subquery optimizations. ○ Optimized derived tables and views. ○ Execution control. ○ Optimizer control. ○ EXPLAIN improvements. Less I/O, CPU, memory requirements. Faster execution.
- 20. © SkySQL Ab. Company Confidential. MariaDB 10 Performance: More Enhancements and Features ● Fusion-io Atomic Writes - improves performance on popular SSD hardware.✔ exclusive to MariaDB 10 ● XtraDB storage engine - enhanced from InnoDB. ● Improved InnoDB storage engine - based on MySQL 5.6. ● Performance schema - real-time performance management. ● Improved thread pool - better concurrent user performance.✔ exclusive to MariaDB 10 ● More optimizer improvements - higher efficiency in I/O accesses. ✔ exclusive to MariaDB 10 Faster performance, lower cost.
- 21. © SkySQL Ab. Company Confidential. Fusion-IO page compression https://blog.mariadb.org/significant-performance-boost-with-new-mariadb-page-compression-on-fusionio • Atomic writes gives a performance increase of about 30%. By enabling fast checksum for XtraDB it’s 50% • By using page compression the compression ratio is leading to better performance and there are less writes to disk. • Multi-threaded flush provides better throughput and decreases operation latencies delivering a performance boost
- 22. © SkySQL Ab. Company Confidential. MariaDB 10 Interoperability: CONNECT Storage Engine ● From 3rd party developer. ● Maps diverse data to tables. ● JOIN mapped data to DB tables. ● Flat files including CSV. ● Tables in external DBs. ● Generated tables (PIVOT etc.) ● Plug-in API for your own mappings. Powerful tool for data integration, federation. Application Spi der MariaDB Parser/Optimizer/Connection Pool CONNECT Engine Other Engines MySQL MariaDB ✘ ✔ in 10.0. Technology Preview Database Tables .log XML CSV
- 23. © SkySQL Ab. Company Confidential. MariaDB 10 Interoperability: Cassandra Storage Engine ● Window into a Cassandra ring: read/write like a table in MariaDB. ● Use standard SQL queries. ● JOIN Cassandra data to MariaDB tables. ● Use a MariaDB cluster for high-availability access. ● Bring data from Cassandra into OLTP applications. Interoperate with Cassandra. Use Cassandra data in OLTP applications. Application Spi der Database Tables MariaDB Parser/Optimizer/Connection Pool Cassandra Engine Other Engines MySQL MariaDB ✘ ✔ in 10.0. Technology Preview
- 24. © SkySQL Ab. Company Confidential. MariaDB 10 NoSQL Features: Dynamic Columns ● Store unstructured data in MariaDB tables with a simple API. ● Use MariaDB’s indexing and transactions to manipulate “document” style data fast and consistently. ● Nest sets of dynamic columns inside of other dynamic columns - hierarchical structuring. ● Include multiple rows with dynamic columns in transactions. Process unstructured data in the same way as NoSQL,but with the power of MariaDB. Cust ID Account Balance Dyn_Col_BLOBs 2035 $154.04 NAME: John Smith|LOC: 45.35243, -74.98348|IMAGE: x27A8B8C ... 2036 $929.10 NAME: Jane Doe|LOC: 45.35243, -74.98348|AGE: 32| GENDER: F... 2037 $377.53 NAME: Carol Jones|AGE: 43|GENDER: F||IMAGE: xA9674DE678 ... MySQL MariaDB ✘ ✔ in 10.0.
- 25. © SkySQL Ab. Company Confidential. MariaDB 10 NoSQL Features: HandlerSocket Plugin ● Simple, low-overhead API direct to the storage engine. ● Bypasses SQL statement processing. ● Less CPU, memory, I/O, network demands. ● Batches requests for even less I/O operations. ● Building block for application level NoSQL processing. Mix RDBMS applications with high performance NoSQL applications. Leverage MariaDB’s storage engine architecture for both. Transactional Application Database Tables NoSQL Application Spi der Connection Pool InnoDB/XtraDB and Spider Storage Engines SQL Interface Parser Optimizer HandlerSocket Plugin SQL Statements Simple CRUD API MySQL MariaDB ✘ ✔ in 10.0.
- 26. © SkySQL Ab. Company Confidential. Thread Time 27 .003761 28 34.2529 29 .000713 Why is this so slow? Lets find out… SHOW EXPLAIN FOR 28; ID select_type table possible_ keys rows 1 SIMPLE tbl NULL 1855174 Oh! Sequential scan of a 18M row table… no wonder! I’ll let the team know... MariaDB 10 Operations: SHOW EXPLAIN Command To: DevOps Team From: DBA Debby Subj. Slow Queries Hey guys, figured out those blocking queries. The new reservation app is looking for the shortest route but that column has no index - we need one! I’ll add it now. Debby MySQL MariaDB ✘ ✔ in 10.0. And it works in the slow query log too! Also for UPDATE and DELETE.
- 27. © SkySQL Ab. Company Confidential. MariaDB 10 Operations: More Enhancements and Features ● Improved Table Discovery - less work for DBAs; integrates storage engines. ✔ exclusive to MariaDB 10 ● SHOW PLUGINS SONAME - dynamic report on installed components. ✔ exclusive to MariaDB 10 ● SHUTDOWN Command - apps can shutdown the database programmatically. ✔ exclusive to MariaDB 10 ● Kill Query by Query ID - fix performance blocks . ● Online ALTER TABLE - schema changes without downtime. ● Per-thread Memory Statistics - easier tuning and problem diagnosis. ✔ exclusive to MariaDB 10 ● Improved Error Messages - faster problem solving.✔ exclusive to MariaDB 10 Manage larger configurations with lower cost, less downtime.
- 28. © SkySQL Ab. Company Confidential. Database Tables MariaDB 10 Role: DBA Permissions: ● Update Schema ● View Statistics ● Create Database DBA Developer Sysadmin MariaDB 10: Security: Role-Based Access Control MySQL MariaDB ✘ ✔ in 10.0.
- 29. © SkySQL Ab. Company Confidential. MariaDB 10 Security: More Enhancements and Features ● Audit Plugin - Identify and correct potential security breaches. Comply with auditability requirements. ✔ included with MariaDB 10, supported by SkySQL ● PAM Authentication Module - delegate authentication to the popular Unix single-signon authentication API.✔ exclusive to MariaDB 10 Integrate the database with your security policies. Simplify administration and compliance.
- 30. © SkySQL Ab. Company Confidential. MariaDB Audit Plugin 2.0 Roadmap ● Audit Plugin 1.1 includes table event logging such as triggers and stored procedure calls, unique to its integration with MariaDB. ● Building on these capabilities, V2.0 of the Audit Plugin is planned to include: ○ Optional field substitution of placeholders in query logs to improve privacy and security. ○ Integration with MariaDB 10 Role-based Access Control - filtering audit logs by role. ○ Recording of privilege changes as audit-able events. ○ Log rotation. Confidential
- 31. © SkySQL Ab. Company Confidential. MariaDB 10 Roadmap 10.0.X Releases - updates to 10.0 over 2014. • Merge additional MySQL 5.6 changes. • Improvements to Fusion-io SSD atomic write support. • More thread pool improvements including Percona enhancements. • SHOW EXPLAIN support for PARTITIONS and EXTENDED attributes. • CONNECT engine improvements. • Perform extensive replication testing between MySQL 5.6 and MariaDB 10.0 to establish a more complete supportability matrix. • Open Query Graph (OQGRAPH) engine improvements. • Experimental Mroonga storage engine - full-text for Asian languages. Confidential
- 32. © SkySQL Ab. Company Confidential. MariaDB 10 Roadmap 10.1.X Preliminary Major Feature List (all dot.dot versions): • Integration of Galera multi-master clusters into MariaDB • Switch on/off Galera support in conf file • Merged features from MySQL, Percona, WebscaleSQL: • All functions visible from SQL in 5.6. • Character set and collation changes. • default_tmp_storage_engine. • Optimizer features including semi-joins+outer-joins, optimizer trace, EXPLAIN JSON, EXPLAIN FOR CONNECTION. • Percona features: per-query variables, userstat improvements, SHOW ENGINE INNODB STATUS improvements, NUMA improvements. • Selected 5.7 features: UNION ALL without temporary tables, multiple triggers on table, new InnoDB, new Performance Schema instruments, MDL improvements, multiple triggers per table, new InnoDB/XtraDB. • Merge WebscaleSQL improvements and integrate stress-test. • Kerberos authentication support • GIS improvements - full support for OGC compliance • Windowing functions Confidential
- 33. © SkySQL Ab. Company Confidential. MariaDB 10 Roadmap 10.1.X Preliminary Major Feature List, continued: • Performance boosts: thread management, 16K InnoDB buffers, remote and portable tablespaces, optimizer enhancements, temporary table improvements, InnoDB persistent auto-increment, query timeouts. • Performance tuning Parallel Replication and GTID, row-level PR enhancements, slave execution of triggers, more benchmarks. • Setting to force InnoDB tables to have a primary key. • Role-based access control: features and performance improvements. • Progress report on mysqldump • Memory engine varchar and BLOB types • Community: move version control to GitHub (DONE). Confidential
- 34. © SkySQL Corporation Ab. Company Confidential. MariaDB 10 Roadmap 10.1 and following - additional feature ideas under investigation: • Transactional DDL • Indexes on expressions • Field/Column type plug-in • JSON/BSON • YAML • IPv6/IPv4 • UUID • Own datatype for Dynamic Columns Confidential • More storage engines and interoperability with other DBs - potentially: • RocksDB/LevelDB • InfiniDB • MongoDB • Hadoop/HDFS • HBase/Honeycomb • Message queues • Global DDL lock
- 35. © SkySQL Corporation Ab. Company Confidential. MariaDB Enterprise 03/07/2014 35
- 36. © SkySQL Corporation Ab. Company Confidential. MariaDB Enterprise: Integrating the Complete Solution 36 MariaDB Enterprise Subscriptions DevOps Admin NoSQL Sharding Clustering Innovation Support Consulting Training RDBA Portal NRE Customer CarePartner Products
- 37. © SkySQL Corporation Ab. Company Confidential. MariaDB Enterprise: What’s In A Subscription? Benefit What you get beyond the core MariaDB server binaries Documentation Customer portal includes subscription-only material beyond KnowledgeBase. Connectors Tested by SkySQL, certified with ISVs. Product Roadmaps Roadmaps shared with subscribers, who have stronger influence on direction. Upgrades, Patches Subscribers receive proactive upgrade push, with release notes and advice. Helpdesk Support Subscribers receive consultative support and hot-fixes, beyond break/fix. Bundled Partner Tools Selected partner tools offered as part of subscription.