MySQL Cluster - Latest Developments (up to and including MySQL Cluster 7.4)

MySQL Cluster
Dive into the Latest Developments
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Andrew Morgan
@andrewmorgan
clusterdb.com
Bernhard Ocklin
bernd.ocklin@oracle.com
2nd October 2014

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.

Today’s Database Requirements
Extreme Write Scalability Real-Time User Experience
Rock Solid Reliability Rapid Service Innovation

Today’s Database Requirements
Transactional Integrity
OLTP + Real-Time Analytics
Extreme Write Scalability Real-Time User Experience
Standards & Skillsets
Rock Solid Reliability Rapid Service Innovation

Who’s Using MySQL Cluster

MySQL Cluster Overview
• Auto-Sharding, Multi-Master
• ACID Compliant, OLTP + Real-Time Analytics
HIGH SCALE, READS +
WRITES
• Shared nothing, no Single Point of Failure
• Self Healing + On-Line Operations
99.999% AVAILABILITY
• In-Memory Optimization + Disk-Data
• Predictable Low-Latency, Bounded Access Time
REAL-TIME
• Key/Value + Complex, Relational Queries
• SQL + Memcached + JavaScript + Java + HTTP/REST & C++
SQL + NoSQL
• Open Source + Commercial Editions
• Commodity hardware + Management, Monitoring Tools
LOW TCO

MySQL Cluster Architecture
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer

MySQL Cluster Scaling
Clients
Application Layer
Data Layer

MySQL Cluster HA
Clients
Application Layer
Data Layer

Scale
On-Line
Operations
Backup
Evolve
Upgrade

MySQL Cluster 7.3: 1.2 Billion UPDATEs per Minute
• NoSQL C++ API,
flexaSynch benchmark
• 30 x Intel E5-2600 Intel
Servers, 2 socket, 64GB
• ACID Transactions, with
Synchronous Replication
25
20
15
10
5
0
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
Millions of UPDATEs per Second

NoSQL Access to MySQL Cluster data
Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps
JPA
Cluster JPA
PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached
MySQL JNI Node.JS mod_ndb ndb_eng
NDB API (C++)

Cluster & Memcached – Schema-Free
key value
<town:maidenhead,SL6>
key value
Key Value
town:maidenhead SL6
generic table
Application view
SQL view

Cluster & Memcached - Configured Schema
key value
prefix key value
Prefix Table Key-col Val-col policy
town: map.zip town code cluster
Config tables
town ... code ...
maidenhead ... SL6 ...
map.zip
Application view
SQL view

Real-Time Analytics - Adaptive Query Localization
• Scale Complex Queries across Shards
• JOINs pushed down to data nodes
• Executed in parallel
• Returns single result set to MySQL
• Opens Up New Use-Cases
• Real-time analytics
• Recommendations engines
• Fraud-detection
mysqld
Data Nodes
mysqld
A
Q
L
Data Nodes
DON’T COMPROMISE
FUNCTIONALITY TO SCALE-OUT !!

Query Execution Time Seconds
87.23 seconds
MySQL Cluster 7.1 MySQL Cluster 7.2
Real-World Test Case
100
90
80
70
60
50
40
30
20
10
0
• Web-Based Content Management System
– JOINs 11-tables, 33.5k rows
– Returns 2k rows, 19 columns per row
1.26 seconds
70x
Faster

 Foreign Key Support
 MySQL 5.6
 Auto-Installer
 NoSQL JavaScript for node.js

MySQL Cluster 7.3: Foreign Keys
• Brings MySQL Cluster to a broader range of workloads
– Packaged apps, custom projects
• Adds powerful functionality while reducing complexity
– App logic & data model
• Enabled by default
• Enforced for SQL &
NoSQL APIs
• On-line add and drop

MySQL Cluster 7.3: Auto-Installer
• Fast configuration
• Auto-discovery
•Workload optimized
• Repeatable best
practices
Specify
Workload
Auto-
Discover
Define
Deploy Topology

MySQL Cluster 7.3: Connection Thread Scalability
 Increases throughput of each
connection to the data layer
 More client threads can use each
connection by splitting mutexes
Reduces configuration complexity
 Increases cluster scalability
 Each connection consumes one of 256
node ids
Up to 7.5x higher performance per
connection
 SQL & NoSQL interfaces
With Connection
Thread Scalability

MySQL Cluster 7.3: Connection Thread Scalability
 DBT2 Benchmark
– Single MySQL Server
– Single Data Node
– 128 client connections
8.5x

Node.js NoSQL API
• Native JavaScript access to MySQL Cluster
– End-to-End JavaScript: browser to the app & DB
– Storing and retrieving JavaScript objects
directly in MySQL Cluster
– Eliminate SQL transformation
• Implemented as a module for node.js
– Integrates Cluster API library within the web
app
• Couple high performance, distributed apps,
with high performance distributed database
• Optionally routes through MySQL Server
V8 JavaScript Engine
MySQL Cluster Node.js Module
Clients

MySQL Cluster 7.4.1 DMR
Available Now!

Better performance and operational simplicity
• Performance gain over 7.3
– 47% (Read-Only)
– 38% (Read-Write)
• Faster maintenance operations
– Nodal & Rolling restarts
– Upgrades

• Asynchronous replication between
MySQL Clusters
• Active-Active
– Update anywhere
– Conflict detection
• Application notified through exception tables
• Can opt to have conflicts resolved
automatically
– Auto-conflict-resolution
• Conflicting transaction and dependent ones
are rolled-back
• No changes to application schema
Active-Active Geo-Replication

Spend $40 Add $100
What is a conflict?
John.balance==$100
John.balance-=$40
John.balance==$60
John.balance==$200
John.balance==$100
John.balance+= $100
John.balance==$200
John.balance==$600
$60
$200

• Detects conflicting inserts/updates
• Entire transactions (and dependent
ones) rolled back
• All conflicts are handled before
switching primary
Later in MySQL Cluster 7.4
• Conflicting deletes
• Rolling back of transactions that
read conflicted data
Handling of Conflicts

Detecting Conflicts - Reflected GCI
• Primary store logical timestamp (GCI)
against updated row
– Window for conflict opens
• GCI replicated with updated row to
Secondary
• The same row and GCI is replicated back
(reflected) from Secondary to Primary
after it has been applied
– Closing window for conflict
• Primary checks every event originating
from the Secondary to ensure it isn’t for a
‘conflictable’ row
Primary Secondary
John.balance==$100
John.balance-=$40
John.balance==$60
John.balance==$200
John.balance==$100
John.balance+= $100
John.balance==$200
John.balance==$600
Spend
$40
Add $100
$60
$200

How to Use Conflict Detection/Resolution
Decide which
tables need
protecting
For each table,
specify what to
do on conflicts
Just record in
exception table
Application or
DBA acts on
content
Rollback the
conflicting row
Rollback the
conflicting
transaction

Restart Times
What operations benefit?
• Restarting data node with locally checkpointed data
– Major improvement
• Restarting data node which must recover data from peer
– Further speedups to come in 7.4.X (greater parallelization)
• Upgrade/rolling restarts
• Cluster shutdown and restart
– Minor improvement

Enhanced Memory Reporting
mysql> CREATE DATABASE clusterdb;USE clusterdb;
mysql> CREATE TABLE simples (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY) ENGINE=NDB;
mysql> select node_id AS node, fragment_num AS frag, fixed_elem_alloc_bytes alloc_bytes,
fixed_elem_free_bytes AS free_bytes, fixed_elem_free_rows AS spare_rows from
memory_per_fragment where fq_name like '%simples%';
+------+------+-------------+------------+------------+
| node | frag | alloc_bytes | free_bytes | spare_rows |
+------+------+-------------+------------+------------+
| 1 | 0 | 131072 | 5504 | 172 |
| 1 | 2 | 131072 | 1280 | 40 |
| 2 | 0 | 131072 | 5504 | 172 |
| 2 | 2 | 131072 | 1280 | 40 |
| 3 | 1 | 131072 | 3104 | 97 |
| 3 | 3 | 131072 | 4256 | 133 |
| 4 | 1 | 131072 | 3104 | 97 |
| 4 | 3 | 131072 | 4256 | 133 |
+------+------+-------------+------------+------------+
See how much memory a table is using

See how memory is made available after deleting rows
+------+------+-------------+------------+------------+
+------+------+-------------+------------+------------+
| 1 | 0 | 131072 | 5504 | 172 |
| 1 | 2 | 131072 | 1280 | 40 |
| 2 | 0 | 131072 | 5504 | 172 |
| 2 | 2 | 131072 | 1280 | 40 |
| 3 | 1 | 131072 | 3104 | 97 |
| 3 | 3 | 131072 | 4256 | 133 |
| 4 | 1 | 131072 | 3104 | 97 |
| 4 | 3 | 131072 | 4256 | 133 |
+------+------+-------------+------------+------------+
mysql> DELETE FROM clusterdb.simples LIMIT 1;
+------+------+-------------+------------+------------+
+------+------+-------------+------------+------------+
| 1 | 0 | 131072 | 5504 | 172 |
| 1 | 2 | 131072 | 1312 | 41 |
| 2 | 0 | 131072 | 5504 | 172 |
| 2 | 2 | 131072 | 1312 | 41 |
| 3 | 1 | 131072 | 3104 | 97 |
| 3 | 3 | 131072 | 4288 | 134 |
| 4 | 1 | 131072 | 3104 | 97 |
| 4 | 3 | 131072 | 4288 | 134 |
+------+------+-------------+------------+------------+

Check how well partitioned/sharded a table is
mysql> CREATE TABLE simples (id INT NOT NULL AUTO_INCREMENT, species VARCHAR(20) DEFAULT "Human",
PRIMARY KEY(id, species)) engine=ndb PARTITION BY KEY(species);
// Add some data
mysql> select node_id AS node, fragment_num AS frag, fixed_elem_alloc_bytes alloc_bytes,
fixed_elem_free_bytes AS free_bytes, fixed_elem_free_rows AS spare_rows from ndbinfo.memory_per_fragment
where fq_name like '%simples%';
+------+------+-------------+------------+------------+
+------+------+-------------+------------+------------+
| 1 | 0 | 0 | 0 | 0 |
| 1 | 2 | 196608 | 11732 | 419 |
| 2 | 0 | 0 | 0 | 0 |
| 2 | 2 | 196608 | 11732 | 419 |
| 3 | 1 | 0 | 0 | 0 |
| 3 | 3 | 0 | 0 | 0 |
| 4 | 1 | 0 | 0 | 0 |
| 4 | 3 | 0 | 0 | 0 |
+------+------+-------------+------------+------------+

Oracle MySQL HA & Scaling Solutions
MySQL
Replication
MySQL
Fabric
Oracle VM
Template
Oracle
Clusterware
Solaris
Cluster
Windows
Cluster
DRBD
MySQL
Cluster
App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔
Platform Support All All Linux Linux Solaris Windows Linux All
Clustering Mode
Master +
Slaves
Master +
Slaves
Active/Pas
sive
Active/Passi
ve
Active/P
assive
Active/Pas
sive
Active/P
assive
Multi-
Master
Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec
Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔
Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔
Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔
Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔
Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB
Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔
37

When to Consider MySQL Cluster
• Scalability demands
– Sharding for write performance?
• Latency demands
– Cost of each millisecond?
• Uptime requirements
– Cost per minute of downtime?
– Failure versus maintenance?
• Application agility
– Developer languages and frameworks?
– SQL or NoSQL?

MySQL Cluster Auto-Installer
• Fast configuration
• Auto-discovery
•Workload optimized
• Repeatable best practices
Specify
Workload
Auto-
Discover
Define
Deploy Topology

Enhancing DevOps Agility, Reducing Downtime
MySQL Cluster Manager
Automated Management
• Start / Stop node or
whole cluster
• On-Line Scaling
• On-Line Reconfiguration
• On-Line Upgrades
• On-Line Backup &
Restore
• Import Running Cluster
Self-Healing
• Node monitoring
• Auto-recovery extended
to SQL + mgmt nodes
HA Operations
• Cluster-wide
configuration
consistency
• Persistent
configurations
• HA Agents

Upgrade using MCM
Application Layer
Data Layer
upgrade cluster --package=7.3 mycluster;

MySQL Cluster Manager 1.3.2 GA
Available Now!

MySQL Cluster Manager 1.3.2 GA
“Unmanaged” production Cluster
mcm> create cluster --import
mcm> import config [--dryrun]
mcm> import cluster[--dryrun]
Cluster now managed by MCM
Import a running Cluster into MCM

Oracle University MySQL Training Services
Prepare Your Organization to Enable Reliable and High-Performance Web-Based Database Applications
Top Courses for Administrators and Developers
Top Certifications
“Training and team skill
have the most significant impact on overall performance of
technology and success of technology projects.” - IDC, 2013
Premier Support customers eligible to
save 20% on learning credits.
Benefits
 Expert-led training to support your MySQL learning needs
 Flexibility to train in the classroom or online
 Hands-on experience to gain real world experience
 Key skills needed for database administrators and developers
• MySQL for Beginners
MySQL for Database Administrators
MySQL Performance Tuning
MySQL Cluster – NEW - Register Your Interest!
MySQL and PHP - Developing Dynamic Web Applications
MySQL for Developers
MySQL Developer Techniques
MySQL 5.6 Database Administrator
MySQL 5.6 Developer
To find out more about available MySQL Training & Certification
offerings, go to: education.oracle.com/mysql
RECENTLY RELEASED
!!ALL NEW!! MySQL Cluster Training
To Register your interest to influence the
schedule on this newly released course – go to
education.oracle.com/mysql and click on the
MySQL Cluster Course

Next Steps
Learn More
• www.mysql.com/cluster
• Authentic MySQL Curriculum: http://oracle.com/education/mysql
Try it Out
• dev.mysql.com/downloads/cluster/
Let us know what you think
• bugs.mysql.com
• forums.mysql.com/list.php?25

MySQL Cluster - Latest Developments (up to and including MySQL Cluster 7.4)

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to MySQL Cluster - Latest Developments (up to and including MySQL Cluster 7.4) (20)

More from Andrew Morgan (8)

Recently uploaded (20)

MySQL Cluster - Latest Developments (up to and including MySQL Cluster 7.4)