What's new in MariaDB Platform X3
0 likes577 views
The document outlines the new features and improvements in MariaDB, including enhancements to performance, security, and temporal capabilities across different versions such as 10.3 and 10.4. Key updates include instant operations for altering tables, an enhanced authentication system with data-at-rest encryption, and the introduction of application-time period tables for tracking valid data changes. Additionally, it discusses high availability features with support for group commits and streaming replication in the MariaDB cluster.
![Instant ALTER TABLE
Adding Columns
| ADD [COLUMN] [IF NOT EXISTS] col_name column_definition
[FIRST | AFTER col_name ]
| ADD [COLUMN] [IF NOT EXISTS] (col_name column_definition,...)
● In MariaDB Server 10.3 a column needs to be added as the last column
MariaDB [10.3]> alter table t1 add column mychar3 char(2) FIRST;
ERROR 1845 (0A000): ALGORITHM=INSTANT is not supported for this operation. Try
ALGORITHM=INPLACE](https://image.slidesharecdn.com/whatisnew-190306001652/85/What-s-new-in-MariaDB-Platform-X3-13-320.jpg)
![Instant ALTER TABLE
Changing Column Order
| CHANGE [COLUMN] [IF EXISTS] old_col_name new_col_name column_definition
[FIRST|AFTER col_name]
● With MariaDB 10.4 FIRST / AFTER are supported as instant operation
○ Data type and size cannot be changed](https://image.slidesharecdn.com/whatisnew-190306001652/85/What-s-new-in-MariaDB-Platform-X3-14-320.jpg)
![Instant ALTER TABLE
Changing Column Order
| MODIFY [COLUMN] [IF EXISTS] col_name column_definition
[FIRST | AFTER col_name]
● Instant if
○ Data type and size does not change](https://image.slidesharecdn.com/whatisnew-190306001652/85/What-s-new-in-MariaDB-Platform-X3-15-320.jpg)
![Instant ALTER TABLE
Dropping a Column
| DROP [COLUMN] [IF EXISTS] col_name [RESTRICT|CASCADE]
● Instant removal of a column](https://image.slidesharecdn.com/whatisnew-190306001652/85/What-s-new-in-MariaDB-Platform-X3-16-320.jpg)
![Optimizer Trace
● JSON object recording execution
path
○ Understand why the optimizer is
choosing a path
mariadb> set optimizer_trace=1;
mariadb> <query>;
mariadb> select * from
information_schema.optimizer_trace;
{
"steps": [
{
"join_preparation": {
"select#": 1,
"steps": [
{
"expanded_query": "/* select#1 */ select
`t1`.`col1` AS `col1`,`t1`.`col2` AS `col2` from `t1`
where (`t1`.`col1` < 4)"
}
]
}
},
{
"join_optimization": {
"select#": 1,
"steps": [
{
"condition_processing": {
"condition": "WHERE",
"original_condition": "(`t1`.`col1` < 4)",
...](https://image.slidesharecdn.com/whatisnew-190306001652/85/What-s-new-in-MariaDB-Platform-X3-21-320.jpg)
What's new in MariaDB Platform X3
- 1. MARIADB What’s New Max Mether VP MariaDB Server PM MariaDB Corporation
- 3. MariaDB TX MariaDB Server MariaDB MaxScale InnoDB/MyRocks MariaDB AX MariaDB Server MariaDB MaxScale ColumnStore App/dev MariaDB TX BI/data science MariaDB AX
- 4. MariaDB TX 3.0 MariaDB Server 10.3 MariaDB MaxScale 2.2 InnoDB/MyRocks MariaDB AX 2.0 MariaDB Server 10.2 MariaDB MaxScale 2.2 ColumnStore 1.2 MariaDB Platform X3 MariaDB MaxScale 2.3 MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3
- 5. Applications Containers MariaDB Platform X3 MariaDB MaxScale 2.3 CDC MariaDB Server 10.3 InnoDB/MyRocks MariaDB Server 10.3 ColumnStore 1.3 Transactional Analytical Kubernetes (Helm) Docker (Compose) C JDBC ODBC Node.js Ingest streaming data Kafka connector Administration SQL Diagnostic Manager SQLyog MariaDB Backup MariaDB Flashback Import bulk data Spark connector C/Java/Python API
- 6. MariaDB Server (Spider) Application Connection 1 MariaDB Server 1 Row storage MariaDB Server 2 Row storage MariaDB Server n Row storage MariaDB Server (ColumnStore) CS node 1 Columnar storage CS node 2 Columnar storage CS node n Columnar storage Sharding Distributed storage MariaDB MaxScale Transactional Analytical
- 8. MariaDB Server 10.4 Performance Security Temporal High Availability
- 9. WHAT WE’RE FOCUSING ON 1. New Instant operations to speed up altering tables for InnoDB 2. Enhancing the security features in regards to a. User Accounts b. Data-at-Rest 3. Implement the next level for working with temporal tables 4. Setting the ground for providing more features for MariaDB Cluster 5. Better Backup
- 10. Instant Schema Changes For InnoDB Performance Security Temporal High AvailabilityPerformance
- 11. Overview ● Instant operations for altering tables ○ Speed up operations ○ Lower storage space requirements for ALTER TABLE operations ● Adding columns including reordering ● Drop columns ● reorder columns ○ FIRST, LAST, AFTER ● Instant charset and collation changes ● Increasing maximum column length for VARCHAR
- 12. Instant ALTER TABLE set alter_algorithm=instant; | ALGORITHM = INSTANT ● Can also added directly to the query ● The algorithm INSTANT results in an error, if ○ The table would need a full copy ○ Data would be partially copied ○ Data would need to be read Assure an Instant Operation
- 13. Instant ALTER TABLE Adding Columns | ADD [COLUMN] [IF NOT EXISTS] col_name column_definition [FIRST | AFTER col_name ] | ADD [COLUMN] [IF NOT EXISTS] (col_name column_definition,...) ● In MariaDB Server 10.3 a column needs to be added as the last column MariaDB [10.3]> alter table t1 add column mychar3 char(2) FIRST; ERROR 1845 (0A000): ALGORITHM=INSTANT is not supported for this operation. Try ALGORITHM=INPLACE
- 14. Instant ALTER TABLE Changing Column Order | CHANGE [COLUMN] [IF EXISTS] old_col_name new_col_name column_definition [FIRST|AFTER col_name] ● With MariaDB 10.4 FIRST / AFTER are supported as instant operation ○ Data type and size cannot be changed
- 15. Instant ALTER TABLE Changing Column Order | MODIFY [COLUMN] [IF EXISTS] col_name column_definition [FIRST | AFTER col_name] ● Instant if ○ Data type and size does not change
- 16. Instant ALTER TABLE Dropping a Column | DROP [COLUMN] [IF EXISTS] col_name [RESTRICT|CASCADE] ● Instant removal of a column
- 17. INSTANT ALTER TABLE Increasing column length for VARCHAR ALTER TABLE t1 CHANGE f1 f1 VARCHAR(256), alter_algorithm=instant; ALTER TABLE t1 MODIFY f1 VARCHAT(256), alter_algorithm=instant; ● Depends on the used InnoDB row format ○ Default is DYNAMIC ● Any extension of VARCHAR from ≤127 bytes ● Unlimited if ROW_RORMAT=REDUNDANT
- 19. Overview ● New Optimizer defaults ● Optimizer Trace ○ JSON object recording the execution path through the optimizer ● Condition push down ○ into materialized IN subqueries ○ From HAVING into WHERE ● Histograms based on random row samples ● In-memory primary key / rowid filters
- 20. New Optimizer Defaults ● Make use of histogram information and engine independent table statistics by default ○ Condition Selectivity ■ Use selectivity of all range predicates estimated with histogram ○ Use engine independent table statistics (EITS) ○ Build histograms when collecting EITS ● Auto-size for the join buffer ● Use index statistics (cardinality) instead of records_in_range for large IN-lists
- 21. Optimizer Trace ● JSON object recording execution path ○ Understand why the optimizer is choosing a path mariadb> set optimizer_trace=1; mariadb> <query>; mariadb> select * from information_schema.optimizer_trace; { "steps": [ { "join_preparation": { "select#": 1, "steps": [ { "expanded_query": "/* select#1 */ select `t1`.`col1` AS `col1`,`t1`.`col2` AS `col2` from `t1` where (`t1`.`col1` < 4)" } ] } }, { "join_optimization": { "select#": 1, "steps": [ { "condition_processing": { "condition": "WHERE", "original_condition": "(`t1`.`col1` < 4)", ...
- 23. Overview ● Enhanced Authentication and Privilege System ● Improvements for Data-at-Rest ● Enhancements to Security Maintenance For MariaDB Enterprise Server ● Audit Plugin Enhancements* ● Galera Data-at-Rest encryption for Gcache*
- 24. Enhanced Authentication / Privilege System ● Password Expiration ○ To fulfill enhanced security requirements ● Disable User Accounts via SQL ○ Allows to temporarily lock an account ● SET PASSWORD statement ○ Simplifies changing a password ● Enable authentication via unix_socket by default ○ mysql_secure_installation will ask for setup unix_socket based authentication ● Multiple Authentication Methods per User ○ Allows to verify a user internally if an external authentication is currently not available ● Block user accounts after a number of failed login attempts
- 25. Improvements for Data-at-Rest ● Key rotation now also for the InnoDB Redo Log ○ It was a limitation for the redo log that key rotation was not possible ● Full Data-at-Rest Encryption also for MariaDB Cluster (Enterprise Feature) ○ The Galera Cluster Gcache (IST Transaction Log) is now encrypted ○ Encryption enabled by using binary log Data-at-Rest encryption ● Spatial Index for InnoDB is now providing Data-at-Rest encryption
- 26. Enhancements to Security Maintenance ● Dynamic loading of SSL certificates ○ SSL certificates can be reloaded without a server restart ○ Certificate revocation list (CRL) reload ● Server log includes information about ignored passwords ○ Passwords are ignored if external authentication is used ● New format for the user table for future enhancements
- 27. Audit Plugin Enhancements ● Enhanced internal API ○ When writing a plugin, more information is available ● Enterprise Audit Plugin ○ Flexible User based filter definition for auditing ■ Auditing requirements for human users and application users can be different ○ Filters and templates defined in system tables ○ Auditing configuration options and changes can be logged ○ Connections can be filtered
- 28. Temporal Application-Time Period Tables Performance Security Temporal High Availability Temporal
- 29. Overview ● MariaDB Server 10.3 introduced System Versioned Tables ○ Tracking data changes based on “system time of change” ○ Special requirement for auditing ● NEW: Application-Time Period Tables ○ Tracking data changes based on the time, a change is valid for ○ Application controlled timestamps ○ Limitation: The option WITHOUT OVERLAPS is not yet supported
- 30. Compare Type of Temporal ● Time of change by system time ● Forensic analysis & legal requirements ○ Store data for N years. ● Data analytics (retrospective, trends etc.) ● Point-in-time recovery - recover as of particular point in time System Versioned ● Valid time period defined by application ● Insurance Applications ● Banking (transfer limit, ...) ● whenever data has to be valid for a given start and end date “period of interest” Application-Time Period Tables
- 31. Bitemporal Application-Time Period + System Versioned Tables combined in one Table
- 32. Application-Time Period Tables ● With Application-Time Period Tables the timestamps of the period are defined by the user/application ○ Period of validity defined and changed by INSERT, UPDATE acc_num Start End maxTrans 1234 2010-01-01 2011-11-12 5000 acc_num Start End maxTrans 1234 2010-01-01 2011-11-12 5000 1234 2011-02-03 2011-09-10 10000 1234 2011-09-10 2011-11-12 5000 UPDATE Emp FOR PORTION OF accPeriod FROM DATE '2011-02-03' TO DATE '2011-09-10' SET maxTrans = 10000 WHERE acc_num = 1234;
- 33. High Availability Enhancements to MariaDB Cluster Performance Security Temporal High AvailabilityHigh Availability
- 34. Overview ● MariaDB Cluster based on Galera Version 4 ○ New Galera 4 based API in MariaDB Server will allow further extensions ○ New Galara 4 Library ● Support of MariaDB group commits ● Streaming Replication for huge transactions ● Support of Instant Alter for InnoDB ● Rolling Upgrade from MariaDB Cluster 10.3 to 10.4
- 35. Support of MariaDB group commits ● MariaDB Server group commits allow parallel replication ○ Galera now also can gain advantage of the grouped transactions ● A group of transactions can be committed as a block ● Performance advantages where non-conflicting transactions exist
- 36. Streaming Replication ● Size of a transaction limited in MariaDB Cluster 10.3 ● No limitation with streaming replication ○ Recommendation: Huge transactions still should be an exception ● Transaction replicated in fragments Huge Transaction Support
- 37. Optimized Backup
- 38. Overview ● Optimized “FLUSH TABLES WITH READ LOCK” (FTWRL) ○ Instant FTWRL for not used tables ○ Second FTWRL for the rest of the tables when not used anymore ● Implementation of backup stages ○ Engine aware locking methods ● BACKUP LOCK statement ○ Meta lock on a table to block DDL ○ Allows consistent copy of dependent files ● Enterprise Backup tool to create consistent backups while minimizing needed locks ○ DDL allowed to execute in parallel ○ Locking minimized for all engine
- 39. And even more ...
- 40. Overview ● Crash safe system tables by using Aria ● Validation for field type JSON ○ The JSON_VALID constraint will be added by default ● Unique indexes are now available also for blobs ● Support of brackets (parentheses) in UNION/EXCEPT/INTERSECT
- 41. THANK YOU!