![I have went a step forward and created a “Master to Master” relationship in which 1 database is
both the master and the slave and the other is also the master and the slave. This means if a
change is made to on the live it will instantly be applied to the backup server. Also if a change is
made on the backup server then it will be replicated instantly on the live so its a “2way”
replication.
setting up a master to master relationship
I have this article to set up the master to master relationship:
https://www.digitalocean.com/community/tutorials/howtosetupmysqlmastermasterreplicatio
n
If you are working Centos Linux distribution user “server_id” instead of “serverid” and make
changes under the [mysqld] line. Also the configuration file in Centos is usually in /etc/my.cnf if
not enter command locate i my.cnf to find it.
Troubleshooting MySQL replication
Steps 2 and 3 in the article above may need to be reapplied if servers lose connection.
make sure the you can enter as “replicator” from live to backup and from backup to live
For example, in the live server type this command # mysql u [replicator username]
p[replicator password] h[backup server IP/domain] it should log you in..
make sure firewall on backup server does not block incoming connections on tcp port
3306 (mysql port) from the live server and vice versa.
login to mysql and type “show slave status” and read any error messages carefully and
maybe try googling it.
Do not forget to test replication changes on both servers after resolving issues.
“Unison” 2way file synchronization
I have installed and configured “Unison” as the file synchronization program. This is a 2way file
synchronization program and works similarly to the “Master to Master” relationship described
above except that is related to files and changing their content not databases. Here is the article
I used to set both servers on:
http://xmodulo.com/synchronizefilesbetweentwoservers.html
Page 2 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-2-320.jpg)
![# Optionally, you can sync specific sub directories only (under the root).
path = test_project1/uploads/
path = test_project2/uploads/
path = test_project3/uploads/
# Optionally, you can ignore specific files or directories that are matched with
regular expressions.
# ignore = Name *.o
# ignore = Name *~
# ignore = Path */temp/archive_*
# If you want to ignore difference in file props:
perms = 0
#suppress message: Contacting Server
#contactquietly = true
#show text interface instead of graphic
ui = text
Some important notes to mention on the file…
root = /var/www/
root = ssh://ogsis@50.63.52.151//var/www/
The first line the local root path and the second is server root path which should be in this format
ssh://[username]@[host]/[absolute path to the root]. You need to have keybased passwordless
ssh access to the server (which means access the server without entering its password) read
this article here if you did not do this already or just need to understand it:
https://www.centosblog.com/passwordlesssshkeybasedauthentication60secondscentoslin
ux/
path = test_project1/uploads/
path = test_project2/uploads/
path = test_project3/uploads/
These are the sub directories that will be checked which mostlikely will be the uploaded files.
#show text interface instead of graphic
ui = text
Page 4 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-4-320.jpg)
![# if any of the 2 files don’t exist then consider them as “on”
if [ f $server_status_file ]; then
exists=true;
else
echo "on" > $server_status_file;
fi
#check if ssh service in the server is running, value must be "off" or "on"
(anything other the "off" is considered "on")
if [ f $ssh_service_file ]; then
exists=true;
else
echo "on" > $ssh_service_file;
fi
if [ "$(cat $server_status_file)" != "off" ] && [ "$(cat $ssh_service_file)" != "off" ];
then
echo "unison ui text sync_live" | at t `date d '+1 minute' "+%Y%m%d%H%M"`;
echo "./unison_sync.bash" | at t `date d '+1 minute' "+%Y%m%d%H%M"`;
date > last_unison_status.txt;
fi
One note to mention…
server_status_file="./server_status.txt";
ssh_service_file="./server_ssh_service.txt";
The script basically checks if one these files has “off” as their content and if it does not then it
will schedule to run unison the next 1 minute and places a timestamp in a text file when the last
run was schedule which is 1 minute ahead of the time stamp.
Page 6 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-6-320.jpg)
![You can easily stop scheduling by changing the content of 1 of the 2 files mentioned above to
“off” and the next minute will be unison will not run. You can then start it after stopping it by just
running by first making sure script is executable by running this command:
chmod +x ./unison_sync.bash
Then making sure that BOTH files mentioned above are set to anything BUT “off” or do not exist
(as they will created with “on” as their content if they do not exist.
Then running the script:
./unison_sync.bash
These files are controlled by the monitoring script as well which is mention in the next section..
Monitoring script:
We need to be vigilant on issues that occur and notified immediately if anything happens. The
backup server will have a constant eye on the live server. I have made a monitoring scripts that
runs every minute and checks 4 things the “nmap” (which must must installed before hand)
command:
If the the server is running or if connection to the server from the backup could not be
made
if the the apache web server is running (by checking tcp port 80)
if the MySQL database server is running (tcp port 3306)
if the SSH server is running (tcp port 22) which is important for unison to be running
Requirements before running
Make sure that the backup server has access to all ports mentioned above as well as the server
itself you can use this command to check the status:
# nmap p 22,80,3306 [server_address]
If you do not have access to the server or to 1 or more of the ports mentioned above please
check the incoming connections on the firewall on the live server.
The script will use the mail command so make sure you can send an email from the BACKUP
server by entering this command:
Page 7 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-7-320.jpg)
![#!/bin/bash
#vars
server_address="192.168.0.129";
control_file="./control.txt";
exists=false;
#check if control file exists
if [ f $control_file ]; then
exists=true;
else
echo "on" > $control_file;
fi
#first save output to file
nmap p 22,80,3306 $server_address > last_nmap_status.txt
#check if server is down
cat last_nmap_status.txt | grep i "Nmap done:" |
while read r line
do
#split by open parenthesis "("
IFS=" " read a sep_field <<< $line;
received=${sep_field[5]//(};
if [ $received lt 1 ]; then
echo "off email was sent.." > control.txt ;
echo "off" > server_status.txt;
subject=$server_address" is DOWN!";
body="Here is the nmap status:
"$(cat last_nmap_status.txt);
echo "$subject
Page 9 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-9-320.jpg)
![";
echo "$body";
echo "$body" | mail s "$subject" $USER;
exit 0;
fi
done
# check apache server
cat last_nmap_status.txt | grep i "80/tcp" |
while read r line
do
#split by open parenthesis "("
IFS=" " read a sep_field <<< $line;
port_check=${sep_field[1]};
if [ $port_check == "closed" ]; then
echo "off email was sent.." > control.txt ;
subject=$server_address" apache/web service is DOWN!";
body="Here is the nmap status:
"$(cat last_nmap_status.txt);
echo "$subject
";
echo "$body";
echo "$body" | mail s "$subject" $USER;
fi
done
# check mysql server
cat last_nmap_status.txt | grep i "3306/tcp" |
while read r line
do
Page 10 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-10-320.jpg)
![#split by open parenthesis "("
IFS=" " read a sep_field <<< $line;
port_check=${sep_field[1]};
if [ $port_check == "closed" ]; then
echo "off email was sent.." > control.txt ;
subject=$server_address" MySQL/Database service is DOWN!";
body="Here is the nmap status:
"$(cat last_nmap_status.txt);
echo "$subject
";
echo "$body";
echo "$body" | mail s "$subject" $USER;
fi
done
# check ssh server
cat last_nmap_status.txt | grep i "22/tcp" |
while read r line
do
#split by open parenthesis "("
IFS=" " read a sep_field <<< $line;
port_check=${sep_field[1]};
if [ $port_check == "closed" ]; then
echo "off email was sent.." > control.txt ;
echo "off" > server_ssh_service.txt;
subject=$server_address" ssh service is DOWN!";
body="Here is the nmap status:
"$(cat last_nmap_status.txt);
Page 11 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-11-320.jpg)
![echo "$subject
";
echo "$body";
echo "$body" | mail s "$subject" $USER;
fi
done
date > last_check_nmap_run.txt;
if [ "$(cat $control_file)" == "on" ]; then
echo "./check_nmap.bash" | at t `date d '+1 minute' "+%Y%m%d%H%M"`;
fi
“GoDaddy” A records Domain configuration
According to dnsimple.com the “A record” is:
“The A in A record stands for Address. Simply put, an A record is used to find the
address of a computer connected to the internet from a name. Whenever you visit a
web site, send an email, connect to Twitter or Facebook or do almost anything on the
Internet, the address you enter is a series of words connected with dots.
For example, to get to this blog you may have followed a link from another web site
pointing to blog.dnsimple.com. In my DNSimple account is an A record that points
blog.dnsimple.com to the IP address 72.32.231.8.”
http://support.dnsimple.com/articles/arecord/
We will use the”A records” to have a domain automatically redirect from live to backup instantly
when the live server goes down.
viewing “A records” for a domain on GoDaddy
Page 12 out of 20](https://image.slidesharecdn.com/457e80ba-181c-4d29-b033-8cea5e10a474-150803102957-lva1-app6892/85/DRPTechnicalDocumentationV1-0-12-320.jpg)
DRPTechnicalDocumentationV1.0
- 1. DRP Technical Documentation V1.0 October 14, 2014 By: Azaam Alfi Summary: Our goal here is to synchronize all files which include code and other files such as uploads and images necessary to run to all projects which are web applications and MySQL data in real time between the live server and the backup server. This is required incase our live server goes down or loses data. Our backup server will need to be hot all the time which means that anytime we need to close the live server and use the backup server it need to be up and running with the most current version of data and files. There are 4 major components I have worked on that you MUST understand: MySQL native data replication “Unison” 2way file synchronization Monitoring script “GoDaddy” A records Domain configuration MySQL native data replication: Data replication here is for the 2 identical databases in the live and backup server. Basically replication here is applying any change made to 1 database or database server instantly to the other database or the database server. For example, one record is added to table “A” in the live server. That same record is instantly added to table “A” in the backup server through the MySQL replication. Replication has a “Master” database and a “Slave” database the changes made on the “Master” are applied on the “Slave” but the changes made on the “Slave” are not applied to the “Master”. Having 1 database as “Master” and the other as “Slave” is called a “Master to Slave” relationship. You can learn more by clicking these links: http://en.wikipedia.org/wiki/Replication_%28computing%29#Database_replication http://dev.mysql.com/doc/refman/5.0/en/replication.html Page 1 out of 20
- 2. I have went a step forward and created a “Master to Master” relationship in which 1 database is both the master and the slave and the other is also the master and the slave. This means if a change is made to on the live it will instantly be applied to the backup server. Also if a change is made on the backup server then it will be replicated instantly on the live so its a “2way” replication. setting up a master to master relationship I have this article to set up the master to master relationship: https://www.digitalocean.com/community/tutorials/howtosetupmysqlmastermasterreplicatio n If you are working Centos Linux distribution user “server_id” instead of “serverid” and make changes under the [mysqld] line. Also the configuration file in Centos is usually in /etc/my.cnf if not enter command locate i my.cnf to find it. Troubleshooting MySQL replication Steps 2 and 3 in the article above may need to be reapplied if servers lose connection. make sure the you can enter as “replicator” from live to backup and from backup to live For example, in the live server type this command # mysql u [replicator username] p[replicator password] h[backup server IP/domain] it should log you in.. make sure firewall on backup server does not block incoming connections on tcp port 3306 (mysql port) from the live server and vice versa. login to mysql and type “show slave status” and read any error messages carefully and maybe try googling it. Do not forget to test replication changes on both servers after resolving issues. “Unison” 2way file synchronization I have installed and configured “Unison” as the file synchronization program. This is a 2way file synchronization program and works similarly to the “Master to Master” relationship described above except that is related to files and changing their content not databases. Here is the article I used to set both servers on: http://xmodulo.com/synchronizefilesbetweentwoservers.html Page 2 out of 20
- 7. You can easily stop scheduling by changing the content of 1 of the 2 files mentioned above to “off” and the next minute will be unison will not run. You can then start it after stopping it by just running by first making sure script is executable by running this command: chmod +x ./unison_sync.bash Then making sure that BOTH files mentioned above are set to anything BUT “off” or do not exist (as they will created with “on” as their content if they do not exist. Then running the script: ./unison_sync.bash These files are controlled by the monitoring script as well which is mention in the next section.. Monitoring script: We need to be vigilant on issues that occur and notified immediately if anything happens. The backup server will have a constant eye on the live server. I have made a monitoring scripts that runs every minute and checks 4 things the “nmap” (which must must installed before hand) command: If the the server is running or if connection to the server from the backup could not be made if the the apache web server is running (by checking tcp port 80) if the MySQL database server is running (tcp port 3306) if the SSH server is running (tcp port 22) which is important for unison to be running Requirements before running Make sure that the backup server has access to all ports mentioned above as well as the server itself you can use this command to check the status: # nmap p 22,80,3306 [server_address] If you do not have access to the server or to 1 or more of the ports mentioned above please check the incoming connections on the firewall on the live server. The script will use the mail command so make sure you can send an email from the BACKUP server by entering this command: Page 7 out of 20
- 9. #!/bin/bash #vars server_address="192.168.0.129"; control_file="./control.txt"; exists=false; #check if control file exists if [ f $control_file ]; then exists=true; else echo "on" > $control_file; fi #first save output to file nmap p 22,80,3306 $server_address > last_nmap_status.txt #check if server is down cat last_nmap_status.txt | grep i "Nmap done:" | while read r line do #split by open parenthesis "(" IFS=" " read a sep_field <<< $line; received=${sep_field[5]//(}; if [ $received lt 1 ]; then echo "off email was sent.." > control.txt ; echo "off" > server_status.txt; subject=$server_address" is DOWN!"; body="Here is the nmap status: "$(cat last_nmap_status.txt); echo "$subject Page 9 out of 20
- 10. "; echo "$body"; echo "$body" | mail s "$subject" $USER; exit 0; fi done # check apache server cat last_nmap_status.txt | grep i "80/tcp" | while read r line do #split by open parenthesis "(" IFS=" " read a sep_field <<< $line; port_check=${sep_field[1]}; if [ $port_check == "closed" ]; then echo "off email was sent.." > control.txt ; subject=$server_address" apache/web service is DOWN!"; body="Here is the nmap status: "$(cat last_nmap_status.txt); echo "$subject "; echo "$body"; echo "$body" | mail s "$subject" $USER; fi done # check mysql server cat last_nmap_status.txt | grep i "3306/tcp" | while read r line do Page 10 out of 20
- 11. #split by open parenthesis "(" IFS=" " read a sep_field <<< $line; port_check=${sep_field[1]}; if [ $port_check == "closed" ]; then echo "off email was sent.." > control.txt ; subject=$server_address" MySQL/Database service is DOWN!"; body="Here is the nmap status: "$(cat last_nmap_status.txt); echo "$subject "; echo "$body"; echo "$body" | mail s "$subject" $USER; fi done # check ssh server cat last_nmap_status.txt | grep i "22/tcp" | while read r line do #split by open parenthesis "(" IFS=" " read a sep_field <<< $line; port_check=${sep_field[1]}; if [ $port_check == "closed" ]; then echo "off email was sent.." > control.txt ; echo "off" > server_ssh_service.txt; subject=$server_address" ssh service is DOWN!"; body="Here is the nmap status: "$(cat last_nmap_status.txt); Page 11 out of 20