MariaDB High Availability
Download as PPTX, PDF1 like2,657 views
This document discusses different approaches to achieving high availability with MariaDB databases, including replication, Galera clusters, and MaxScale load balancing. It describes asynchronous and semi-synchronous replication topologies that provide redundancy and enable failover. Synchronous replication with Galera clusters is discussed as another option that ensures all nodes remain in sync with no data loss. MaxScale is presented as a tool that can monitor MariaDB topologies and perform load balancing and failover across nodes.
MariaDB High Availability
- 2. MariaDB MaxScaleMariaDB Multi-Master Cluster O P E R A T I N G S Y S T E M / F I L E S Y S T E M / S A N / C L O U D Application Connectors MariaDB Server NoSQL CRUD API Original Core MariaDB MariaDB Engineering Community Contribution Replicas Supporting Asynchronous, Semi-Sync & Synchronous replication MariaDB C JDBC ODBC Replication Kernel Production Plugins Parallel Slave GTIDBinLog API Multi-Source SQL Parser Cache/Buffer Optimiser Connection Pool Temporal PL/SQL Audit AWS KMS Authentication Handler Socket Etc. 40+ Plugins SQL Lightweight Transactional Interoperability Performance & Scalability Graph & SearchAnalytics InnoDB XtraDBAria Memory MyISAM CONNECTColumnStoreSpider OQGRAPH MyRocks Mroonga STORAGE LAYER EXTENSIBILITY KERNEL EXTENSIBILITY
- 3. High Availability Defined In information technology, high availability refers to a system or component that is continuously operational for a desirably long length of time. Availability – Wikipedia up time / total time
- 4. Approach to HA 3.7 days / year Backup / Restore 1 < 99.9% 52.6 min / year Replication / Automatic failover 3 ~ 99.99% 8.8hs / year Simple replication / manual failover 2 ~ 99.9% 5.3 min / year Galera Cluster ~ 99.999% 4 5 Other Strategies for High Availability
- 5. An average of 80 percent of mission-critical application service downtime is directly caused by people or process failures. The other 20 percent is caused by technology failure, environmental failure or a disaster Gartner Research
- 6. High Availability Background • High Availability isn’t always equal to long Uptime – A system is “up” but it might not be accessible – A system that is “down” just once, but for a long time, is NOT highly available • High Availability rather means – Long Mean Time Between Failures (MTBF) – Short Mean Time To Recover (MTTR) • High availability is: – a system design protocol and associated implementation that ensures a certain degree of operational continuity during a given measurement period.
- 7. High Availability Components High availability is a system design protocol and associated implementation that ensures a certain degree of operational continuity during a measurement period. For stateful services, we need to make sure that data is made redundant. It is not a replacement for backups! Data Redundancy Some mechanism to redirect traffic from the failed server or Datacenter to a working one Failover or Switchover Solution Availability of the services needs to be monitored, to take action when there is a failure or even to prevent them Monitoring and Management
- 9. General Terms • Single Point of Failure (SPOF) – An element is a SPOF when its failure results in a full stop of the service as no other element can take over (storage, WAN connection, replication channel) – It is important to evaluate the costs for eliminating the SPOF, the likelihood that it may fail, and the time required to bring it into service again • Downtime – the period of time a service is down. Planned and unplanned. Planned downtime is part of the overall availability • Shared vs. Local Storage – Shared storage systems like SANs can provide built-in high availability, though this comes with equally high costs – Not really suitable for Disaster Recovery scenario on multiple Data Center – Local storage comes with low cost but we need to implement ways for replication/mirroring
- 10. General Terms • Switchover – When a manual process is used to switch from one system to a redundant or standby system in case of a failure • Failover – Automatic switchover, without human intervention • Failback – A (often-underestimated) task to handle the recovery of a failed system and how to fail-back to the original system after recovery
- 12. Replication Scheme All nodes are masters and applications can read and write from any node Synchronous Replication The Master does not confirm transactions to the client application until at least one slave has copied the change to its relay log, and flushed it to disk Semi-Syncronous Replication The Master does not wait for Slave, the master writes events to its binary log and slaves request them when they are ready Asynchronous Replication
- 13. HA Begins from Data Replication • Replication enables data from one MariaDB server (the master) to be replicated to one or more MariaDB servers (the slaves). • MariaDB Replication is: – very easy to setup – used to scale out read workloads – provide a first level of high availability and geographic redundancy – offload backups and analytic jobs.
- 14. Asynchronous Replication • MariaDB Replication is asynchronous by default. • Slave determines how much to read and from which point in the binary log • Slave can be behind master in reading and applying changes. – Single threaded vs parallel replication • If the master crashes, transactions might not have been transmitted to any slave • Asynchronous replication is great for read scaling as adding more replicas does not impact replication latency
- 15. Asynchronous Replication-Switch Over 1. The master server is taken down or we encounter a fault by our monitoring 2. The slave server is updated to the last position in the relay log 3. The clients point at the designated slave server 4. The designated slave server becomes the master server 5. All steps are manual Master and Slaves ReadOnly Slaves Master and Slaves ReadOnly Slaves
- 16. Async Replication Topologies Master and Slaves ReadOnly Slaves Master with Relay Slave Circular Replication
- 17. Semi-synchronous Replication • MariaDB supports semi-synchronous replication: – the master does not confirm transactions to the client application until at least one slave has copied the change to its relay log, and flushed it to disk. – In semi-synchronous replication, only after the events have been written to the relay log and flushed does the slave acknowledge receipt of a transaction's events – Semi-synchronous is a practical solution for many cases where high availability and no data-loss is important. – When a commit returns successfully, it is known that the data exists in at least two places (on the master and at least one slave). – Semi- synchronous has a performance impact due to the additional round trip
- 18. MariaDB Enhanced Semi-synchronous Replication • One or more slaves can be defined as working semi-synchronously. • For these slaves, the master waits until the I/O thread on one or more of the semi-synch slaves has flushed the transaction to disk. • This ensures that all committed transactions are at least stored in the relay log of the slave. • If no semi-synch slave can acknowledge the transaction, the master will downgrade to asynchronous replication after waiting for a timeout period. Once a semi-synch slave comes back online, the master will reset back to semi-synch replication.
- 19. Semi-synchronous Replication – Switch Over • The steps for a failover are the same as when using the standard replication • but in Step 2, a slave should be chosen among those (if many) that are be semi- synched with the master Master and Slaves Semi-Sync Slave Async Slaves Master and Slaves Async Slaves
- 20. Semi-Sync Replication Topologies • Semi- synchronous replication is used between master and backup master • Semi- sync replication has a performance impact, but the risk for data loss is minimized. • This topology works well when performing master failover – The backup master acts as a warm-standby server – it has the highest probability of having up-to-date data if compared to other slaves. Semi_sync Asynchronous ReadOnly/ Backup Master ReadOnly
- 21. MariaDB Multi-Source Replication • It enables a slave to receive transactions from multiple sources simultaneously. • It can be used to backup multiple servers to a single server, to merge table shards, and consolidate data from multiple servers to a single server. Master 2Master 1 Master 3 Slave
- 22. Combining MariaDB Replication Features • Replication features can be combined to form more resilient configurations • Example: – Implement semi-sync circular replication to increase data resilience – Use GTID to avoid duplicate transactions – Use read-only slaves for read scale out – Use MaxScale: • Transactions will go to active master • Reads will be offloaded to slaves • Fast failover Semi_sync Asynchronous Backup Master ReadOnly
- 23. Synchronous Replication (Galera) • Galera Replication is a synchronous multi-master replication plug-in that enables a true master-master setup for InnoDB. • Every component of the cluster (node) is a share nothing server • All nodes are masters and applications can read and write from any node • A minimal Galera cluster consists of 3 nodes: – A proper cluster needs to reach a quorum (i.e. the majority of the nodes of the cluster) • Transactions are synchronously committed on all nodes. MariaDB MariaDB MariaDB
- 24. Synchronous Replication (Galera) • PROS – A high availability solution with synchronous replication, failover and resynchronization – No loss of data – All servers have up-to-date data (no slave lag) – Read scalability – 'Pretty good' write scalability MariaDB MariaDB MariaDB
- 25. Synchronous Replication (Galera) • CONS – It only supports InnoDB – The transaction rollback rate and hence the transaction latency, can increase with the number of the cluster nodes – The cluster performs as its least performing node • an overloaded master affects the performance of the Galera cluster – Network latency affects transaction throughput MariaDB MariaDB MariaDB
- 26. MaxScale for HA
- 27. MDBE Cluster Failover Clustered nodes cooperate to remain in sync With multiple master nodes, reads and updates both scale* Synchronous replication with optimistic locking delivers high availability with little overhead Fast failover because all nodes remains synchronizedMariaDB MariaDB MariaDB Load Balancing and Failover Application / App Server
- 28. MaxScale Use Case Master/Slaves Async Replication MaxScale monitors a MariaDB Topology Master/Slaves + R/W split routing Max Scale MariaDB
- 29. MaxScale Use Case Master/Slaves Async Replication Master/Slaves + R/W split routing Max Scale MariaDB 1 1 . Master failure
- 30. MaxScale Use Case Master/Slaves Async Replication 1 . Master failure 2 . MaxScale Monitor detects the master_down event Master/Slaves + R/W split routing Max Scale MariaDB script Failover Script master_down event 2
- 31. MaxScale Use Case Master/Slaves Async Replication 1 . Master failure 2 . MaxScale Monitor detects the master_down event 3 . In case it is configured, MaxScale launches a Failover Script that promotes a slave as a new Master Master/Slaves + R/W split routing Max Scale MariaDB script Failover Script master_down event 2 Promote as master3
- 32. MaxScale Use Case Master/Slaves Async Replication 1 . Master failure 2 . MaxScale Monitor detects the master_down event 3 . In case it is configured, MaxScale launches a Failover Script that promotes a slave as a new Master Master/Slaves + R/W split routing Max Scale MariaDB script Failover Script master_down event 2 Promote as master3
- 33. MaxScale Use Case Master/Slaves Async Replication 1 . Master failure 2 . MaxScale Monitor detects the master_down event 3 . In case it is configured, MaxScale launches a Failover Script that promotes a slave as a new Master 4 . MaxScale monitor automatically detects new replication topology after the switch Master/Slaves + R/W split routing Max Scale MariaDB 2 4
- 34. MaxScale Use Case MDBE Cluster Synchronous Replication Each application server uses only 1 connection MaxScale selects one node as “master” and the other nodes as “slaves” If the “master” node fails, a new one can be elected immediately Galera Cluster + R/W split routing Max Scale
- 35. MariaDB HA: MaxScale • Re-route traffic between master and slave(s) • Does not manage servers • Failover / slave promotion is an external process • Implemented for Booking.com • Part of MaxScale release • All slaves are in sync, easy to promote any slave tter Detects Active Master Binary Log Server
- 36. HA / Scalability with MaxScale 2.1 Existing in MaxScale 2.0 New in MaxScale 2.1 Aurora Cluster Monitor Multi-master and Failover Mode for MySQL Monitor Read-write Splitting with Master Pinning Transaction Scaling to support user growth and simplify applications MariaDB Master/Slave and MariaDB Galera Cluster – Load balancing – Database aware dynamic query routing – Traffic profile based routing Replication Scaling to support web-scale applications’ user base Binlog Server for horizontal scaling of slaves in Master/Slave architecture Multi-tenant database scaling to transparently grow tenants and data volume Schema sharding Connection Rate Limitation
- 37. Thank you
- 38. Vielen Dank!
- 39. Thank you Speaker’s Name Speaker’s email