Mysql S&M

by Rob Kaufman [email_address] MySQL S&M

Slaves and Masters What is Master Slave Replication One database instance (the Master) that does both reads and writes This guy is in charge Should do mostly writing One or more database instances that only do reads (the Slaves) Only know what the Master tells them Never except writes

Master Slave Diagram Master Slave 1 Slave 2 Slave 3 App Server App Server Key: Arrow represents data flow direction

Master Slave: Why What will a Master Slave setup do for you? Redundancy Hot spares of data Any slave can become the master if the master fails Efficiency If you data is read bound Many reads to a few writes

Master Slave: Why Not What draw backs does a Master Slave setup have Not a backup! Your backing up your data... right? If not, go do it now Yes I really mean it... get up and go NOW Won't help the write bound Many web apps are about creating data, not just consuming it

Master Slave: How Create slave user GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%.mydomain.com' IDENTIFIED BY 'slavepass'; Configure Master Configure Slave Copy The Data Use Innodb

Be the Master Master Configuration my.cnf: [mysqld] log-bin=mysql-bin server-id=1 Watch out for the skip-networking option Make a data backup mysqldump --all-databases --lock-all-tables --master-data >dbdump.db

Be the Slave Slave Setup Don't have to do log-bin, but you should my.cnf: [mysqld] log-bin=mysql-bin server-id=2 master-host = host_ip_address master-user = replication master-password = slavepass master-port = 3306

Whip that Slave Import the data Start slave with --skip-slave option mysql < dbdump.db Start slave with mysql command "start slave;" check with "show slave status;" and "show master status;"

Slaves in Rails How do I do this in Rails Masochism Hey, I didn't pick the name ;-) Plugin by Rick Olsen piston import http://ar-code.svn.engineyard.com/plugins/masochism/ vendor/plugins add master_database entry into your database.yml set production entry to point at slave add to production.rb config.after_initialize do ActiveReload::ConnectionProxy.setup! end

Slaves in Rails See Robby Russell for more info http://www.robbyonrails.com/articles/2007/11/15/master-slave-databases-with-ruby-on-rails

Other Slave in Rails Options Active Delegate Multiple Database Connections in Ruby on Rails From Robby Russell (http://www.robbyonrails.com/articles/2007/10/05/multiple-database-connections-in-ruby-on-rails) Likely abandoned magic_multi_connections From Dr Nic (http://drnicwilliams.com/2007/04/12/magic-multi-connections-a-facility-in-rails-to-talk-to-more-than-one-database-at-a-time/) This gem is also able to handle master/slave setups but also could be used to do data partitioning across multiple databases. Admittedly has issues

Other Slave in Rails Options ActsAsReadonlyable From the Revolution Health team (http://revolutiononrails.blogspot.com/2007/04/plugin-release-actsasreadonlyable.html) DynamicDatabaseChanger Rails Database Loadbalancing http://rubyforge.org/projects/ddcplugin/ May work for multi-master set ups too (http://scriptserver.blogspot.com/2007/06/rails-database-loadbalancing-with.html) Does not do failover if instance goes down

Master Master Replication What is Master Master Replication All Master database instances can both read and write They are set up in a "circle" that passes data from one instance to another Every master is one machines slave Every slave is another machines master Can be mixed with Master Slave Each Master has multiple slaves One slave of each master is the next master in the chain

Master Master Diagram Master 2 Master 1 Master 3 App Server App Server Key: Arrow represents data flow direction App Server

Master Master: Why What will a Master Master setup do for you? Redundancy Hot spares of data If a master fails it can be removed from the loop its app servers can be redirected elsewhere data will stop moving around the circle till it is fixed or removed Efficiency If you are either write or read bound Can scale cleanly up to 10 instances w/ fast connection between servers

Master Master: Why Not What drawbacks does a Master Master setup have Not a backup! Can propagate DROP and DELETE just as fast as it can UPDATE and INSERT Can get behind It is possible that the data might not all be propagated to every instance at a given moment If a db instance goes down If network latency is really high If traffic is higher than the db instance can handle

Master Master: How Create slave user GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%.mydomain.com' IDENTIFIED BY 'slavepass'; Make a data backup of master1 mysqldump --all-databases --lock-all-tables --master-data >dbdump.db

Be the Master Master 1 configuration my.cnf: [mysqld] log-bin=mysql-bin log-slave-updates replicate-same-server-id = 0 server-id=1 master-host = master2_ip_address master-user = replication master-password = slavepass master-port = 3306 auto_increment_increment = 10 auto_increment_offset = 1

Be the Other Master Master 2 configuration my.cnf: [mysqld] log-bin=mysql-bin log-slave-updates replicate-same-server-id = 0 server-id=2 master-host = master1_ip_address master-user = replication master-password = slavepass master-port = 3306 auto_increment_increment = 10 auto_increment_offset = 2

Whip that... Master? Import the data into master2 Start master2 with --skip-slave option mysql < dbdump.db Start master2 as slave with mysql command "start slave;" Get master2 position with mysql command "show master status" Write down the info under File and Position, use for FILE and POSITION below Back on master1 Start master1 with --skip-slave option Set the master position mysql command CHANGE MASTER TO MASTER_LOG_FILE='FILE', MASTER_LOG_POS=POSITION Start master1 as slave with mysql command "start slave;"

Master Master in Rails Step 1: Setup Master Master Replication for your production enviroment Step 2: Point different clusters of application servers at each different master Step 3: There IS NO STEP 3!

What you want load balancing? We can do load balancing for either Master Slave or Master Master SQLRelay http://sqlrelay.sourceforge.net/documentation.html Stable and well tested solution If one db instance goes down, will server requests to other instances MysqlRelay http://forge.mysql.com/wiki/MySQL_Proxy Kind of the new kid on the block MySQL specific Has other functions besides load balance and failover support

Know When Something is Wrong You need to monitor the replication "show slave status" and "show master status" Munin http://munin.projects.linpro.no/wiki/plugin-mysql_slave_status MMM http://code.google.com/p/mysql-master-master/ PDI

Fix An Error Now what Most of the time what went wrong was the bin log got off by position. This happens sometimes when a db instance crashes or powerfails If so you can use mysql command "STOP SLAVE; SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;" Can be greater than 1 if you want, but watch for danger Sometimes (rarely) it will get ahead Only ever seen this on a disk full Can go back to an old position on the relay log "CHANGE MASTER TO RELAY_LOG_FILE='slave-bin.006', RELAY_LOG_POS=4025;

Thanks Thanks to Robby Russell and Dr Nic They have both done quite a bit of blogging in the Master Slave area Special thanks to RailsMagnet for having links to all the current master-slave plugins for Rails http://railsmagnet.com/2007/12/using-multiple-databases-rails-application MySQL www.mysql.com for its cool Master Master setup Munin http://munin.projects.linpro.no/ for watching my back

Mysql S&M

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to Mysql S&M (20)

More from Rob Kaufman (9)

Recently uploaded (20)

Mysql S&M