SlideShare a Scribd company logo

Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison

Severalnines, profile picture
Severalnines

The document compares MySQL Cluster (NDB) and Galera for MySQL, outlining their architectures, deployment scenarios, and performance monitoring. MySQL Cluster offers high availability through a distributed architecture and asynchronous/synchronous replication, while Galera supports synchronous multi-master replication with guaranteed write consistency. Key considerations include data migration limitations and management, making it crucial for organizations to assess their requirements carefully when choosing between these two solutions.

Technology
1 of 44
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
MySQL Cluster (NDB) vs Galera for MySQL Confidential December 11, 2014 Alex Yu alex@severalnines.com
Copyright Severalnines AB Webinar Housekeeping !This webinar is being recorded !A link to the recording & slides will be posted on severalnines.com !We welcome questions: enter questions into the chat box and we will respond at the end of the presentation !Think of something later? !Email Severalnines at info@severalnines.com 2
Copyright Severalnines AB Agenda !MySQL Cluster (NDB) and Galera Architecture Overview !Read and write workloads !Deployment scenarios !WAN/Geographical replication !Data migration & Schema changes !Management and performance monitoring 3
Copyright Severalnines AB MySQL Cluster (NDB Storage Engine) !Distributed shared nothing realtime database cluster !Multi-master, auto sharding, in-memory & disk-data storage !Near linear scalability with transparent load balancing !SQL and NoSQL interfaces !Local and Geographical Replication !Synchronous and Asynchronous replication !99.999% availability, no single point of failure !Telecom “Carrier Grade” legacy 4
Copyright Severalnines AB MySQL Cluster Applications Example !Subscriber Databases (Telecom HLR/HSS systems) !Massive volume of write traffic (location and updates) !Response time < 3ms !eCommerce !Payment processing and fulfilment !High batch and realtime loads !Service Delivery Platforms !High volume of traffic !Mixed read/write loads 5
Copyright Severalnines AB MySQL Cluster Architecture 6 MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Mgmt MgmNt ode Node Web App Clients NDB API (C++) SQL based clients MySQL Client/Server protocol MGM C API Management Client SQL Nodes Data Nodes Management Nodes (default arbitrator) Synchronous replication within a Node group
Copyright Severalnines AB Automatic Sharding 7 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node Data Node Data Node Data Node
Copyright Severalnines AB Automatic Sharding (cont.) 8 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
Copyright Severalnines AB Automatic Sharding (cont.) 9 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
Copyright Severalnines AB Automatic Sharding (cont.) 10 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
Copyright Severalnines AB Automatic Sharding (cont.) 11 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
Copyright Severalnines AB Automatic Sharding (cont.) 12 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node Data Node Data Node Data Node • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
Copyright Severalnines AB Automatic Sharding (cont.) 13 Data Node Data Node Data Node Data Node 4 partitions Secondary Fragments 4 Data Nodes - 2 Node Groups Primary Fragments 1 2 3 4 • The cluster is fully operational as long as we have 1 node up in each node group! • If all nodes in a single node group is gone then the cluster will gracefully shutdown
Copyright Severalnines AB Automatic Sharding (cont.) 14 Data Node Data Node Data Node Data Node 4 partitions Secondary Fragments 4 Data Nodes - 2 Node Groups Primary Fragments 1 2 3 4 • The cluster is fully operational as long as we have 1 node up in each node group! • If all nodes in a single node group is gone then the cluster will gracefully shutdown
Copyright Severalnines AB Primary Key Requests 15 MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Mgmt MgmNt ode Node Web Web Web • PK lookup goes directly to the node with the primary fragment • Parallel operations, Transparent load balancing
Copyright Severalnines AB Joins, index and table scans 16 MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Mgmt MgmNt ode Node Web Web Web • Table and Index Scans parallel on all nodes • Joins executes on data nodes, merged results sent back to SQL node
Copyright Severalnines AB Migration to MySQL Cluster !Limitations !14K row size, 512 attributes (columns + indexes) / table !32 attributes / key, only first 3072 bytes of column can be used for index !No fulltext or spatial indexes, temporary tables cannot be created using the NDB storage engine !Every table must have a Primary Key !Hidden PK is automatically created if not defined !In-Memory or disk-based tables !Dataset exceeds available system memory for the cluster? !Network, Local and Global Checkpoint !Write intensive, dimension disk subsystem !Dedicated >= 1Gb/s networking !ALTER TABLE … ENGINE NDB !Alt. MySQL Replication, Backup & Restore 17
Copyright Severalnines AB Deployment Scenarios 18 MySQL [SQL Node] MySQL [SQL Node] … Data Node Data Node Data Node Data Node … • Scale up to 48 Data Nodes • Limit of 255 number of nodes (regardless of type)
Copyright Severalnines AB Deployment Scenarios (cont.) 19 Master Slave/Standby MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL Asynchronous Replication Single Point of Failure • Multiple replication topologies available • Master - Master (Bi-directional) West East • Conflict detection and resolution • Master - Slave(s) • Circular • etc Synchronous replication within a Node group
Copyright Severalnines AB Deployment Scenarios (cont.) 20 Master Slave/Standby MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Primary START SLAVE only on Primary! Secondary/Standby West East • Master - Slave(s) • Standby replication channel • Manual failover
Copyright Severalnines AB Deployment Scenarios (cont.) 21 Master Master MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Primary Secondary/Standby • Master - Master (Bi-directional) • Conflict detection and resolution West East • “timestamp based” • row by row not transaction based
Copyright Severalnines AB Galera Cluster for MySQL 22 !Synchronous (Virtually) Multi-Master Replication !Read and Write on any Node !No Master Failover! No Slave Lag! !Guaranteed write consistency !Cluster wide conflicts resolution (certification) !Automatic Node Provisioning !Highly Available and Scalable Client Client Client R/W R/W R/W MySQL [WSREP] !No SPOF !Read and Write (Parallel Applier threads) scalability !Geographical Replication (Mix MySQL Async & Galera Sync) Galera Replication (Synchronous) !Codership, Percona XtraDB Cluster, MariaDB Galera Cluster LB MySQL [WSREP] MySQL [WSREP]
Copyright Severalnines AB Galera Cluster for MySQL (cont.) !Recommended minimum 3 nodes !Network partition/split-brain !Blocking SST (rsync, mysqldump) !Higher probability for “deadlocks” !Cluster wide optimistic locking !Locking conflicts detected at commit !First to commit succeeds !Replication performance dependent on !Network latency !Performance of the “slowest” or the farthest Node (RTT) !Number of deployed nodes 23 Client Client Client R/W R/W R/W MySQL [WSREP] LB MySQL [WSREP] MySQL [WSREP] Galera Replication (Synchronous)
Copyright Severalnines AB Galera Concepts !Primary Component - PC !The whole cluster is a PC during normal operation !Node and network failures ! Splits clusters into several components !Only PC can continue to modify state !Quorum algorithm invoked to select a PC during cluster partitioning !Majority rules !Minority tries to reconnect with PC 24 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Primary Component
Copyright Severalnines AB Galera Concepts (cont.) !State Snapshot Transfer - SST !A transfer of a consistent snapshot of a node state corresponding to a certain GTID !Initialize the state of a newly joining cluster node from an already initialized node (donor) !Incremental State Transfer - IST !Catch up with the cluster by replaying missing transactions ! Known initial node state ! Enough transactions cached at the donor ! gcache.size < database size 25
Copyright Severalnines AB High Latency Network !Galera 2.x WAN replication (MySQL 5.5) !Point to point connection for all nodes! !Transaction latency dependent on the slowest link 26 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] DC1 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] DC2
Copyright Severalnines AB High Latency Network (cont.) !Galera 3.x WAN optimization (MySQL 5.6) !“Cluster” Segment ID to group nodes by location !Replication between segments go over a single connection !Replication writesets distributed within each segment peer to peer !Segment connection/gateway can change per transaction 27 gmcast.segment = 1 gmcast.segment = 2 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] segment gateway DC1 DC2
Copyright Severalnines AB High Latency Network (cont.) !Galera 3.x WAN optimization (MySQL 5.6) !“Cluster” Segment ID to group nodes by location !Replication between segments go over a single connection !Replication writesets distributed within each segment peer to peer !Segment connection/gateway can change per transaction 28 gmcast.segment = 1 segment gateway gmcast.segment = 2 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] DC1 DC2
Copyright Severalnines AB Network Partition/Split Brain !Quorum based system !“Majority >50%” partition continues operation !“Minority” partition blocks operations ! Until reconnected with Primary Component !Use odd number of nodes !Minimum 3 (5, 7, 9 etc) !Galera Arbitrator (garbd) !Useful if you have even number of nodes !Nodes across DCs !Replication relay 29 Galera Arbitrator DC3 MySQL [WSREP] MySQL [WSREP] DC1 MySQL [WSREP] DC2 Client Client Client Load balancer Replication Relay
Copyright Severalnines AB Migration to Galera Cluster !Only InnoDB storage engine !Limited MyISAM support - not recommended !Every Table should have a Primary Key !DELETE operations are unsupported on tables without a primary key !Rows in tables without a primary key may appear in a different order on different nodes. (for cert. md5sum pseudo key from full row) !Transaction size ! A writeset is processed as a single memory-resident buffer ! Extremely large transactions e.g. LOAD DATA can affect performance ! wsrep_load_data_splitting = ON | OFF # 10K inserts/transaction ! wsrep_max_ws_rows (128K), wsrep_max_ws_size (1GB) 30
Copyright Severalnines AB Migration to Galera Cluster (cont.) !Auto Increments !Managed automatically ! Node-1: 1, 4, 7 ! Node-2: 2, 5, 8 ! Node-3: 3, 6, 9 !Auto increment sequence gaps if inserts hit different nodes randomly !Triggers fire only in the Galera node which executes the transaction !Events fire on all nodes 31
Copyright Severalnines AB Schema Changes !DDLs replicated in statement format !Two main methods !TOI - Total Order Isolation !RSU - Rolling Schema Upgrade !wsrep_osu_method = TOI | RSU !wsrep_desync=ON + wsrep_on=OFF !Disconnect from cluster and stop writeset replication (standalone MySQL server) !Dropping Node !Set global wsrep_cluster_address=gcomm:// !Joining must be through IST !Percona Toolkit !pt-online-schema-change 32
Copyright Severalnines AB Schema Changes (cont.) !TOI - Total Order Isolation !Default DDL replication method !Strict consistency, all nodes get the same change !No schema backwards compatibility !Strict commit order force every transactions to wait until DDL is completed !Cluster performance degradation 33
Copyright Severalnines AB Schema Changes (cont.) !RSU - Rolling Schema Upgrade !Desynchronize node from replication until DDL completes !Incoming replication is buffered, nothing is replicated out of the node !After the DDL completes the node will automatically join the cluster and catch up missed transactions from the writeset cache (gcache.size) !Potential no cluster performance degradation !Schema changes need to be backwards compatible ! Applications should be able to use old and new schemas !Only one RSU operation at a time !Rolling operation of the cluster is manual 34
Copyright Severalnines AB Deployment Scenarios 35 Users Users Users HAProxy Load Balancer hthtpttp http http HAProxy Load Balancer R/W R/W R/W MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Galera Replication (Synchronous) ClusterControl hthtpttp Admin VIP http://support.severalnines.com/entries/23612682-Install-HAProxy-and-Keepalived-Virtual-IP- subnet
Copyright Severalnines AB Deployment Scenarios (cont.) 36 Galera as MySQL Slave MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Slave MySQL [Master] MySQL Replication wsrep_mysql_replication_bundle=N • Replication events can be bundled to commit as a single group • Less waits for replication synchronization • wsrep_mysql_replication_bundle=n • Groups n mysql replication transactions in one large transaction
Copyright Severalnines AB Deployment Scenarios (cont.) 37 Galera as MySQL replication Master MySQL [WSREP] Master MySQL [WSREP] MySQL [WSREP] Master MySQL [Slave] MySQL Replication MySQL [Slave] DC1 DC2 • Backups & Reports • Disaster Recovery
Copyright Severalnines AB Deployment Scenarios (cont.) 38 MySQL [WSREP] Disaster Recovery Master Standby MySQL Replication MySQL [WSREP] MySQL [WSREP] Master MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Master DC1 DC2 • “Manual” replication failover • Slave lag
Copyright Severalnines AB Deployment Scenarios (cont.) 39 MySQL [WSREP] Master MySQL [WSREP] Slave MySQL [WSREP] Slave MySQL [Master] MySQL Replication Multi-Source Sink MySQL [Master] MySQL [Slave] http://www.severalnines.com/blog/multi-source-replication-galera-cluster-mysql
Copyright Severalnines AB Severalnines - ClusterControl !Monitor and Manage Heterogeneous Database Cluster !MySQL Cluster, Galera Cluster for MySQL, MongoDB !Automatic !Node and Cluster Recovery !Scheduled Backups !Add/Remove Nodes !Create single DB Node and Cluster !Alerts/Email !Host and DB Metrics 40
Copyright Severalnines AB 41
Copyright Severalnines AB 42
Copyright Severalnines AB 43
Copyright Severalnines AB Thank You! !Severalnines recorded webinars !http://www.severalnines.com/resources/webinars !Severalnines Blog !www.severalnines.com/blog !Galera Cluster for MySQL Intro !http://www.severalnines.com/clustercontrol-mysql-galera-tutorial !MySQL Cluster Training !http://www.severalnines.com/mysql-cluster-training !More Questions? Contact us at: !info@severalnines.com 44

More Related Content

PDF
Best practices for MySQL High Availability
Colin Charles
 
PDF
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Mydbops
 
PPT
Intro to MySQL Master Slave Replication
satejsahu
 
PPTX
Stability Patterns for Microservices
pflueras
 
PDF
M|18 Architectural Overview: MariaDB MaxScale
MariaDB plc
 
PDF
MySQL Database Architectures - MySQL InnoDB ClusterSet 2021-11
Kenny Gryp
 
PDF
Best practices for MySQL High Availability Tutorial
Colin Charles
 
PDF
Apache Kafka - Martin Podval
Martin Podval
 
Best practices for MySQL High Availability
Colin Charles
 
Wars of MySQL Cluster ( InnoDB Cluster VS Galera )
Mydbops
 
Intro to MySQL Master Slave Replication
satejsahu
 
Stability Patterns for Microservices
pflueras
 
M|18 Architectural Overview: MariaDB MaxScale
MariaDB plc
 
MySQL Database Architectures - MySQL InnoDB ClusterSet 2021-11
Kenny Gryp
 
Best practices for MySQL High Availability Tutorial
Colin Charles
 
Apache Kafka - Martin Podval
Martin Podval
 

What's hot (20)

PPTX
From cache to in-memory data grid. Introduction to Hazelcast.
Taras Matyashovsky
 
PDF
[2018] MySQL 이중화 진화기
NHN FORWARD
 
PDF
Best Practice for Achieving High Availability in MariaDB
MariaDB plc
 
PDF
Apache kafka
NexThoughts Technologies
 
PDF
InnoDB Internal
mysqlops
 
PDF
[OpenStack Days Korea 2016] Track1 - All flash CEPH 구성 및 최적화
OpenStack Korea Community
 
PDF
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
The Hive
 
PDF
Automated master failover
Yoshinori Matsunobu
 
PDF
PostgreSQLレプリケーション10周年！徹底紹介！（PostgreSQL Conference Japan 2019講演資料）
NTT DATA Technology & Innovation
 
PDF
MySQL Administrator 2021 - 네오클로바
NeoClova
 
PDF
Oracle Real Application Clusters 19c- Best Practices and Internals- EMEA Tour...
Sandesh Rao
 
PDF
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
Kenny Gryp
 
PDF
MySQL Cluster performance best practices
Mat Keep
 
PDF
MySQL Parallel Replication: inventory, use-case and limitations
Jean-François Gagné
 
PPTX
Introduction to Apache Kafka
Jeff Holoman
 
PDF
Average Active Sessions RMOUG2007
John Beresniewicz
 
PPTX
MySQL Architecture and Engine
Abdul Manaf
 
PDF
The Dual write problem
Jeppe Cramon
 
PDF
MariaDB MaxScale
MariaDB plc
 
PDF
Oracle Active Data Guard 12c: Far Sync Instance, Real-Time Cascade and Other ...
Ludovico Caldara
 
From cache to in-memory data grid. Introduction to Hazelcast.
Taras Matyashovsky
 
[2018] MySQL 이중화 진화기
NHN FORWARD
 
Best Practice for Achieving High Availability in MariaDB
MariaDB plc
 
Apache kafka
NexThoughts Technologies
 
InnoDB Internal
mysqlops
 
[OpenStack Days Korea 2016] Track1 - All flash CEPH 구성 및 최적화
OpenStack Korea Community
 
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
The Hive
 
Automated master failover
Yoshinori Matsunobu
 
PostgreSQLレプリケーション10周年！徹底紹介！（PostgreSQL Conference Japan 2019講演資料）
NTT DATA Technology & Innovation
 
MySQL Administrator 2021 - 네오클로바
NeoClova
 
Oracle Real Application Clusters 19c- Best Practices and Internals- EMEA Tour...
Sandesh Rao
 
MySQL Database Architectures - InnoDB ReplicaSet & Cluster
Kenny Gryp
 
MySQL Cluster performance best practices
Mat Keep
 
MySQL Parallel Replication: inventory, use-case and limitations
Jean-François Gagné
 
Introduction to Apache Kafka
Jeff Holoman
 
Average Active Sessions RMOUG2007
John Beresniewicz
 
MySQL Architecture and Engine
Abdul Manaf
 
The Dual write problem
Jeppe Cramon
 
MariaDB MaxScale
MariaDB plc
 
Oracle Active Data Guard 12c: Far Sync Instance, Real-Time Cascade and Other ...
Ludovico Caldara
 
Ad

Similar to Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison (20)

PDF
MySQL Cluster (NDB) - Best Practices Percona Live 2017
Severalnines
 
PDF
MySQL NDB Cluster 101
Bernd Ocklin
 
PDF
High Availability with MySQL
Thava Alagu
 
PDF
MySQL NDB Cluster 8.0
Ted Wennmark
 
PPT
MYSQL
gilashikwa
 
PDF
Introduction to Galera
Henrik Ingo
 
PDF
MySQL High Availability Solutions
Lenz Grimmer
 
PDF
Mysqlhacodebits20091203 1260184765-phpapp02
Louis liu
 
PDF
MySQL High Availability Solutions
Lenz Grimmer
 
PPTX
My SQL Portal Database (Cluster)
Nicholas Adu Gyamfi
 
ODP
MySQL HA
Kris Buytaert
 
ODP
MySQL Group Replication
Ulf Wendel
 
PDF
Introduction to MySQL Cluster
Abel Flórez
 
PDF
Drupal Con My Sql Ha 2008 08 29
liufabin 66688
 
PDF
OpenStack Days East -- MySQL Options in OpenStack
Matt Lord
 
ODP
MySQL 5.7 clustering: The developer perspective
Ulf Wendel
 
ODP
MySQL Cluster
Abel Flórez
 
ODP
MySQL 5.7 Fabric: Introduction to High Availability and Sharding
Ulf Wendel
 
PPTX
MySQL Options in OpenStack
Tesora
 
PDF
Buytaert kris my_sql-pacemaker
kuchinskaya
 
MySQL Cluster (NDB) - Best Practices Percona Live 2017
Severalnines
 
MySQL NDB Cluster 101
Bernd Ocklin
 
High Availability with MySQL
Thava Alagu
 
MySQL NDB Cluster 8.0
Ted Wennmark
 
MYSQL
gilashikwa
 
Introduction to Galera
Henrik Ingo
 
MySQL High Availability Solutions
Lenz Grimmer
 
Mysqlhacodebits20091203 1260184765-phpapp02
Louis liu
 
MySQL High Availability Solutions
Lenz Grimmer
 
My SQL Portal Database (Cluster)
Nicholas Adu Gyamfi
 
MySQL HA
Kris Buytaert
 
MySQL Group Replication
Ulf Wendel
 
Introduction to MySQL Cluster
Abel Flórez
 
Drupal Con My Sql Ha 2008 08 29
liufabin 66688
 
OpenStack Days East -- MySQL Options in OpenStack
Matt Lord
 
MySQL 5.7 clustering: The developer perspective
Ulf Wendel
 
MySQL Cluster
Abel Flórez
 
MySQL 5.7 Fabric: Introduction to High Availability and Sharding
Ulf Wendel
 
MySQL Options in OpenStack
Tesora
 
Buytaert kris my_sql-pacemaker
kuchinskaya
 
Ad

More from Severalnines (20)

PDF
The Long Term Cost of Managed DBaaS vs Sovereign DBaaS
Severalnines
 
PPTX
Sovereign DBaaS_ A Practical Vision for Self-Implementation of DBaaS.pptx
Severalnines
 
PDF
PostgreSQL on AWS Aurora/Azure Cosmos VS EC2/Azure VMs
Severalnines
 
PDF
Localhost Conference 2024_ Building a Flexible and Scalable Database Strategy...
Severalnines
 
PDF
SREDAY London 2024 | Cloud Native Technologies: The Building Blocks of Modern...
Severalnines
 
PDF
Building a Sovereign DBaaS on K8s OpenInfra Summit Asia 2024.pdf
Severalnines
 
PDF
S-DBaaS Community Call | Introduction to Sovereign DBaaS: The why, what and how
Severalnines
 
PDF
WEBINAR SLIDES: CCX for Cloud Service Providers
Severalnines
 
PPTX
LIVE DEMO: CCX for CSPs, a drop-in DBaaS solution
Severalnines
 
PDF
Kubernetes at Scale: Going Multi-Cluster with Istio
Severalnines
 
PDF
DIY DBaaS: A guide to building your own full-featured DBaaS
Severalnines
 
PDF
Cloud's future runs through Sovereign DBaaS
Severalnines
 
PPTX
Tips to drive maria db cluster performance for nextcloud
Severalnines
 
PPTX
Working with the Moodle Database: The Basics
Severalnines
 
PPTX
SysAdmin Working from Home? Tips to Automate MySQL, MariaDB, Postgres & MongoDB
Severalnines
 
PDF
(slides) Polyglot persistence: utilizing open source databases as a Swiss poc...
Severalnines
 
PDF
Webinar slides: How to Migrate from Oracle DB to MariaDB
Severalnines
 
PDF
Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl
Severalnines
 
PDF
Webinar slides: How to Manage Replication Failover Processes for MySQL, Maria...
Severalnines
 
PDF
Webinar slides: Backup Management for MySQL, MariaDB, PostgreSQL & MongoDB wi...
Severalnines
 
The Long Term Cost of Managed DBaaS vs Sovereign DBaaS
Severalnines
 
Sovereign DBaaS_ A Practical Vision for Self-Implementation of DBaaS.pptx
Severalnines
 
PostgreSQL on AWS Aurora/Azure Cosmos VS EC2/Azure VMs
Severalnines
 
Localhost Conference 2024_ Building a Flexible and Scalable Database Strategy...
Severalnines
 
SREDAY London 2024 | Cloud Native Technologies: The Building Blocks of Modern...
Severalnines
 
Building a Sovereign DBaaS on K8s OpenInfra Summit Asia 2024.pdf
Severalnines
 
S-DBaaS Community Call | Introduction to Sovereign DBaaS: The why, what and how
Severalnines
 
WEBINAR SLIDES: CCX for Cloud Service Providers
Severalnines
 
LIVE DEMO: CCX for CSPs, a drop-in DBaaS solution
Severalnines
 
Kubernetes at Scale: Going Multi-Cluster with Istio
Severalnines
 
DIY DBaaS: A guide to building your own full-featured DBaaS
Severalnines
 
Cloud's future runs through Sovereign DBaaS
Severalnines
 
Tips to drive maria db cluster performance for nextcloud
Severalnines
 
Working with the Moodle Database: The Basics
Severalnines
 
SysAdmin Working from Home? Tips to Automate MySQL, MariaDB, Postgres & MongoDB
Severalnines
 
(slides) Polyglot persistence: utilizing open source databases as a Swiss poc...
Severalnines
 
Webinar slides: How to Migrate from Oracle DB to MariaDB
Severalnines
 
Webinar slides: How to Automate & Manage PostgreSQL with ClusterControl
Severalnines
 
Webinar slides: How to Manage Replication Failover Processes for MySQL, Maria...
Severalnines
 
Webinar slides: Backup Management for MySQL, MariaDB, PostgreSQL & MongoDB wi...
Severalnines
 

Recently uploaded (20)

PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
PDF
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
PPTX
MYSQL Presentation for SQL database connectivity
Swati270511
 
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
PDF
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
PDF
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
PDF
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
IAESIJAI
 
Electronic commerce courselecture one. Pdf
OmerMohamed64
 
MYSQL Presentation for SQL database connectivity
Swati270511
 
Diabetes mellitus diagnosis method based random forest with bat algorithm
IAESIJAI
 
NewMind AI Weekly Chronicles - August'25 Week I
NewMind AI
 
NewMind AI Monthly Chronicles - July 2025
NewMind AI
 
Network Security Unit 5.pdf for BCA BBA.
Serpent6
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
Mobile App Security Testing_ A Comprehensive Guide.pdf
flufftailshop
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 

Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison

  • 1. MySQL Cluster (NDB) vs Galera for MySQL Confidential December 11, 2014 Alex Yu alex@severalnines.com
  • 2. Copyright Severalnines AB Webinar Housekeeping !This webinar is being recorded !A link to the recording & slides will be posted on severalnines.com !We welcome questions: enter questions into the chat box and we will respond at the end of the presentation !Think of something later? !Email Severalnines at info@severalnines.com 2
  • 3. Copyright Severalnines AB Agenda !MySQL Cluster (NDB) and Galera Architecture Overview !Read and write workloads !Deployment scenarios !WAN/Geographical replication !Data migration & Schema changes !Management and performance monitoring 3
  • 4. Copyright Severalnines AB MySQL Cluster (NDB Storage Engine) !Distributed shared nothing realtime database cluster !Multi-master, auto sharding, in-memory & disk-data storage !Near linear scalability with transparent load balancing !SQL and NoSQL interfaces !Local and Geographical Replication !Synchronous and Asynchronous replication !99.999% availability, no single point of failure !Telecom “Carrier Grade” legacy 4
  • 5. Copyright Severalnines AB MySQL Cluster Applications Example !Subscriber Databases (Telecom HLR/HSS systems) !Massive volume of write traffic (location and updates) !Response time < 3ms !eCommerce !Payment processing and fulfilment !High batch and realtime loads !Service Delivery Platforms !High volume of traffic !Mixed read/write loads 5
  • 6. Copyright Severalnines AB MySQL Cluster Architecture 6 MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Mgmt MgmNt ode Node Web App Clients NDB API (C++) SQL based clients MySQL Client/Server protocol MGM C API Management Client SQL Nodes Data Nodes Management Nodes (default arbitrator) Synchronous replication within a Node group
  • 7. Copyright Severalnines AB Automatic Sharding 7 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node Data Node Data Node Data Node
  • 8. Copyright Severalnines AB Automatic Sharding (cont.) 8 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
  • 9. Copyright Severalnines AB Automatic Sharding (cont.) 9 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
  • 10. Copyright Severalnines AB Automatic Sharding (cont.) 10 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
  • 11. Copyright Severalnines AB Automatic Sharding (cont.) 11 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node 1 Data Node 2 Data Node 3 Data Node 4 • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
  • 12. Copyright Severalnines AB Automatic Sharding (cont.) 12 Table T with 8 rows 4 Data Nodes - 2 Node Groups Data Node Data Node Data Node Data Node • Sharding based on hashing the primary key or a user defined key • Each node stores primary fragment for 1 partition and back-up fragment for another • # of node groups == # of data nodes / # of replicas
  • 13. Copyright Severalnines AB Automatic Sharding (cont.) 13 Data Node Data Node Data Node Data Node 4 partitions Secondary Fragments 4 Data Nodes - 2 Node Groups Primary Fragments 1 2 3 4 • The cluster is fully operational as long as we have 1 node up in each node group! • If all nodes in a single node group is gone then the cluster will gracefully shutdown
  • 14. Copyright Severalnines AB Automatic Sharding (cont.) 14 Data Node Data Node Data Node Data Node 4 partitions Secondary Fragments 4 Data Nodes - 2 Node Groups Primary Fragments 1 2 3 4 • The cluster is fully operational as long as we have 1 node up in each node group! • If all nodes in a single node group is gone then the cluster will gracefully shutdown
  • 15. Copyright Severalnines AB Primary Key Requests 15 MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Mgmt MgmNt ode Node Web Web Web • PK lookup goes directly to the node with the primary fragment • Parallel operations, Transparent load balancing
  • 16. Copyright Severalnines AB Joins, index and table scans 16 MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Mgmt MgmNt ode Node Web Web Web • Table and Index Scans parallel on all nodes • Joins executes on data nodes, merged results sent back to SQL node
  • 17. Copyright Severalnines AB Migration to MySQL Cluster !Limitations !14K row size, 512 attributes (columns + indexes) / table !32 attributes / key, only first 3072 bytes of column can be used for index !No fulltext or spatial indexes, temporary tables cannot be created using the NDB storage engine !Every table must have a Primary Key !Hidden PK is automatically created if not defined !In-Memory or disk-based tables !Dataset exceeds available system memory for the cluster? !Network, Local and Global Checkpoint !Write intensive, dimension disk subsystem !Dedicated >= 1Gb/s networking !ALTER TABLE … ENGINE NDB !Alt. MySQL Replication, Backup & Restore 17
  • 18. Copyright Severalnines AB Deployment Scenarios 18 MySQL [SQL Node] MySQL [SQL Node] … Data Node Data Node Data Node Data Node … • Scale up to 48 Data Nodes • Limit of 255 number of nodes (regardless of type)
  • 19. Copyright Severalnines AB Deployment Scenarios (cont.) 19 Master Slave/Standby MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL Asynchronous Replication Single Point of Failure • Multiple replication topologies available • Master - Master (Bi-directional) West East • Conflict detection and resolution • Master - Slave(s) • Circular • etc Synchronous replication within a Node group
  • 20. Copyright Severalnines AB Deployment Scenarios (cont.) 20 Master Slave/Standby MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Primary START SLAVE only on Primary! Secondary/Standby West East • Master - Slave(s) • Standby replication channel • Manual failover
  • 21. Copyright Severalnines AB Deployment Scenarios (cont.) 21 Master Master MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node MySQL [SQL Node] MySQL [SQL Node] Data Node Data Node Data Node Data Node Primary Secondary/Standby • Master - Master (Bi-directional) • Conflict detection and resolution West East • “timestamp based” • row by row not transaction based
  • 22. Copyright Severalnines AB Galera Cluster for MySQL 22 !Synchronous (Virtually) Multi-Master Replication !Read and Write on any Node !No Master Failover! No Slave Lag! !Guaranteed write consistency !Cluster wide conflicts resolution (certification) !Automatic Node Provisioning !Highly Available and Scalable Client Client Client R/W R/W R/W MySQL [WSREP] !No SPOF !Read and Write (Parallel Applier threads) scalability !Geographical Replication (Mix MySQL Async & Galera Sync) Galera Replication (Synchronous) !Codership, Percona XtraDB Cluster, MariaDB Galera Cluster LB MySQL [WSREP] MySQL [WSREP]
  • 23. Copyright Severalnines AB Galera Cluster for MySQL (cont.) !Recommended minimum 3 nodes !Network partition/split-brain !Blocking SST (rsync, mysqldump) !Higher probability for “deadlocks” !Cluster wide optimistic locking !Locking conflicts detected at commit !First to commit succeeds !Replication performance dependent on !Network latency !Performance of the “slowest” or the farthest Node (RTT) !Number of deployed nodes 23 Client Client Client R/W R/W R/W MySQL [WSREP] LB MySQL [WSREP] MySQL [WSREP] Galera Replication (Synchronous)
  • 24. Copyright Severalnines AB Galera Concepts !Primary Component - PC !The whole cluster is a PC during normal operation !Node and network failures ! Splits clusters into several components !Only PC can continue to modify state !Quorum algorithm invoked to select a PC during cluster partitioning !Majority rules !Minority tries to reconnect with PC 24 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Primary Component
  • 25. Copyright Severalnines AB Galera Concepts (cont.) !State Snapshot Transfer - SST !A transfer of a consistent snapshot of a node state corresponding to a certain GTID !Initialize the state of a newly joining cluster node from an already initialized node (donor) !Incremental State Transfer - IST !Catch up with the cluster by replaying missing transactions ! Known initial node state ! Enough transactions cached at the donor ! gcache.size < database size 25
  • 26. Copyright Severalnines AB High Latency Network !Galera 2.x WAN replication (MySQL 5.5) !Point to point connection for all nodes! !Transaction latency dependent on the slowest link 26 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] DC1 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] DC2
  • 27. Copyright Severalnines AB High Latency Network (cont.) !Galera 3.x WAN optimization (MySQL 5.6) !“Cluster” Segment ID to group nodes by location !Replication between segments go over a single connection !Replication writesets distributed within each segment peer to peer !Segment connection/gateway can change per transaction 27 gmcast.segment = 1 gmcast.segment = 2 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] segment gateway DC1 DC2
  • 28. Copyright Severalnines AB High Latency Network (cont.) !Galera 3.x WAN optimization (MySQL 5.6) !“Cluster” Segment ID to group nodes by location !Replication between segments go over a single connection !Replication writesets distributed within each segment peer to peer !Segment connection/gateway can change per transaction 28 gmcast.segment = 1 segment gateway gmcast.segment = 2 MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] DC1 DC2
  • 29. Copyright Severalnines AB Network Partition/Split Brain !Quorum based system !“Majority >50%” partition continues operation !“Minority” partition blocks operations ! Until reconnected with Primary Component !Use odd number of nodes !Minimum 3 (5, 7, 9 etc) !Galera Arbitrator (garbd) !Useful if you have even number of nodes !Nodes across DCs !Replication relay 29 Galera Arbitrator DC3 MySQL [WSREP] MySQL [WSREP] DC1 MySQL [WSREP] DC2 Client Client Client Load balancer Replication Relay
  • 30. Copyright Severalnines AB Migration to Galera Cluster !Only InnoDB storage engine !Limited MyISAM support - not recommended !Every Table should have a Primary Key !DELETE operations are unsupported on tables without a primary key !Rows in tables without a primary key may appear in a different order on different nodes. (for cert. md5sum pseudo key from full row) !Transaction size ! A writeset is processed as a single memory-resident buffer ! Extremely large transactions e.g. LOAD DATA can affect performance ! wsrep_load_data_splitting = ON | OFF # 10K inserts/transaction ! wsrep_max_ws_rows (128K), wsrep_max_ws_size (1GB) 30
  • 31. Copyright Severalnines AB Migration to Galera Cluster (cont.) !Auto Increments !Managed automatically ! Node-1: 1, 4, 7 ! Node-2: 2, 5, 8 ! Node-3: 3, 6, 9 !Auto increment sequence gaps if inserts hit different nodes randomly !Triggers fire only in the Galera node which executes the transaction !Events fire on all nodes 31
  • 32. Copyright Severalnines AB Schema Changes !DDLs replicated in statement format !Two main methods !TOI - Total Order Isolation !RSU - Rolling Schema Upgrade !wsrep_osu_method = TOI | RSU !wsrep_desync=ON + wsrep_on=OFF !Disconnect from cluster and stop writeset replication (standalone MySQL server) !Dropping Node !Set global wsrep_cluster_address=gcomm:// !Joining must be through IST !Percona Toolkit !pt-online-schema-change 32
  • 33. Copyright Severalnines AB Schema Changes (cont.) !TOI - Total Order Isolation !Default DDL replication method !Strict consistency, all nodes get the same change !No schema backwards compatibility !Strict commit order force every transactions to wait until DDL is completed !Cluster performance degradation 33
  • 34. Copyright Severalnines AB Schema Changes (cont.) !RSU - Rolling Schema Upgrade !Desynchronize node from replication until DDL completes !Incoming replication is buffered, nothing is replicated out of the node !After the DDL completes the node will automatically join the cluster and catch up missed transactions from the writeset cache (gcache.size) !Potential no cluster performance degradation !Schema changes need to be backwards compatible ! Applications should be able to use old and new schemas !Only one RSU operation at a time !Rolling operation of the cluster is manual 34
  • 35. Copyright Severalnines AB Deployment Scenarios 35 Users Users Users HAProxy Load Balancer hthtpttp http http HAProxy Load Balancer R/W R/W R/W MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Galera Replication (Synchronous) ClusterControl hthtpttp Admin VIP http://support.severalnines.com/entries/23612682-Install-HAProxy-and-Keepalived-Virtual-IP- subnet
  • 36. Copyright Severalnines AB Deployment Scenarios (cont.) 36 Galera as MySQL Slave MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Slave MySQL [Master] MySQL Replication wsrep_mysql_replication_bundle=N • Replication events can be bundled to commit as a single group • Less waits for replication synchronization • wsrep_mysql_replication_bundle=n • Groups n mysql replication transactions in one large transaction
  • 37. Copyright Severalnines AB Deployment Scenarios (cont.) 37 Galera as MySQL replication Master MySQL [WSREP] Master MySQL [WSREP] MySQL [WSREP] Master MySQL [Slave] MySQL Replication MySQL [Slave] DC1 DC2 • Backups & Reports • Disaster Recovery
  • 38. Copyright Severalnines AB Deployment Scenarios (cont.) 38 MySQL [WSREP] Disaster Recovery Master Standby MySQL Replication MySQL [WSREP] MySQL [WSREP] Master MySQL [WSREP] MySQL [WSREP] MySQL [WSREP] Master DC1 DC2 • “Manual” replication failover • Slave lag
  • 39. Copyright Severalnines AB Deployment Scenarios (cont.) 39 MySQL [WSREP] Master MySQL [WSREP] Slave MySQL [WSREP] Slave MySQL [Master] MySQL Replication Multi-Source Sink MySQL [Master] MySQL [Slave] http://www.severalnines.com/blog/multi-source-replication-galera-cluster-mysql
  • 40. Copyright Severalnines AB Severalnines - ClusterControl !Monitor and Manage Heterogeneous Database Cluster !MySQL Cluster, Galera Cluster for MySQL, MongoDB !Automatic !Node and Cluster Recovery !Scheduled Backups !Add/Remove Nodes !Create single DB Node and Cluster !Alerts/Email !Host and DB Metrics 40
  • 44. Copyright Severalnines AB Thank You! !Severalnines recorded webinars !http://www.severalnines.com/resources/webinars !Severalnines Blog !www.severalnines.com/blog !Galera Cluster for MySQL Intro !http://www.severalnines.com/clustercontrol-mysql-galera-tutorial !MySQL Cluster Training !http://www.severalnines.com/mysql-cluster-training !More Questions? Contact us at: !info@severalnines.com 44