MySQL Live Migration - Common Scenarios
0 likes312 views
1) MySQL live migration involves a 3 step process of taking a baseline dump/snapshot from the source database, setting up replication between the source and destination for continuous data movement, and finally performing a switchover to migrate to the destination. 2) The baseline dump uses mysqldump to export data, users, privileges and stored objects from the source and import to the destination. The --master-data option captures binary log coordinates for setting up replication. 3) Replication is setup using the binary log coordinates to synchronize data between the source and destination. Monitoring replication status ensures the destination lag is low before switchover.
MySQL Live Migration - Common Scenarios
- 1. MySQL Live Migration - Common Scenarios Presented by Prasad Nagaraj
- 2. About ScaleGrid ● Specialize in Database as a Service(DbaaS) solutions ● Support popular open source databases – MongoDB, MySQL, PostgreSQL, Redis ● Multi-cloud support – AWS, Azure, GCP, Digital Ocean, Linode and OCI ● DBA friendly!! ○ Fully managed service automating all routine tasks. You will get ssh access to the underlying machines and root access to the database! ● Developer Friendly!! ○ Slow query analysis, Dev-test clusters, Tools for Online schema changes…. MySQL Live Migration
- 3. Use-cases 1. Migrating to cloud from on-premise 2. Changing managed database service providers 3. Migrating to new hardware 4. Upgrading MySQL versions. MySQL Live Migration
- 4. 3 step process 1. Do a baseline dump/snapshot on source and restore on destination 2. Setup replication between the nodes for continuous data movement 3. Switchover MySQL Live Migration
- 5. Step-1 : Baseline dump ● Use mysqldump utility to export data from source and import to destination ● Move user accounts and privileges • Mysqldump does not support exporting users and privileges. Manually dump Show create user command and Show grants command. • Mysqlpump utility support importing user ● Move data and stored objects • Use the mysqldump options --events, --triggers –routines to export the stored objects ● Save the coordinates of the dump for setting up replication ● Binary log file name and log position co-ordinates are generated when using the –master-data option MySQL Live Migration
- 6. Step-2: Live replication ● Use the snapshot coordinates (binlog file name and position) to setup replication ● Monitor the replication status on the destination using SHOW SLAVE STATUS command(seconds_behind_master, sql_thread and io_thread status) ● Wait for the lag to stabilize close to 5 s or less MySQL Live Migration
- 7. Step-3: Switchover ● Stop writes on the source ● Wait for data to completely sync ● Break the replication from source to destination ● Point your applications to the destination database MySQL Live Migration
- 8. Scenario -1 Ensure Binary logging is enabled on the source. Use --master-data option as part of the mysqldump command to generate binary log coordinates using which replication needs to be setup. MySQL Live Migration Destination Source mysqldump Replication
- 9. Scenario - 2a Use --master-data option as part of the mysqldump command to generate binary log coordinates using which replication needs to be setup. MySQL Live Migration Source (S) (D) Destination Source mysqldump Replication Replica(R)
- 10. Scenario - 2b Use --slave-dump and --include-master-host-port option as part of the mysqldump command ● --slave-dump option provides the binary log coordinates of the ‘source’ of ‘dumped server’ at the time of dump ● --include-master-host-port provides the name and port of the ‘source’ of the ‘dumped server’ to connect to for setting up the replication MySQL Live Migration Source (S) (D) Destination Source mysqldump Replication Replica (R)
- 11. Scenario - 3: GTID compatibility between source and destination ● GTID mode of source and destination servers should be identical ● Example : Replication cannot be setup if source is not using GTID, but destination is. ● In such cases, GTID can be disabled online on the destination using following steps: ( Before setting up the replication) ○ SET GLOBAL gtid_mode = 'ON_PERMISSIVE'; ○ SET GLOBAL gtid_mode = 'OFF_PERMISSIVE'; ○ SET GLOBAL gtid_mode = 'OFF'; ○ SET GLOBAL enforce_gtid_consistency = 'OFF'; MySQL Live Migration
- 12. Scenario - 4: Migrating subset of databases ● mysqldump can be used to import a single or subset of databases from source ● However, binary logs from source will contain changes to all databases on the source including system databases. ● Ensure to set replication filters on the destination so that replication happens only for the databases you are migrating. ● Example: Replication filters can be set online using the command: CHANGE REPLICATION FILTER REPLICATE_WILD_DO_TABLE=('test.%’); MySQL Live Migration
- 13. Scenario - 5 Live migration from RDS ● Baseline dump should be done from read replica ● --slave-dump and --include-master-host-port options cannot be used as they need super privileges which is not available for RDS user ● Replication should be stopped on the read replica before initiating baseline dump. ● Note down SHOW SLAVE STATUS command - Relay_Master_Log_File and Exec_Master_Log_Pos ● Replication should be initiated from RDS source node ● https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Proced ural.Exporting.NonRDSRepl.html MySQL Live Migration
- 14. THANK YOU! Presented by Prasad Nagaraj