NoSQL: Redis High-Availability Cluster Solution and Emergency Maintenance Tips
Premise: There is a master-slave architecture where the master has no persistence configured, and the slave is configured with AOF.
(The master is used for backup and write services, the slave is used to provide read services)
Scenario: What if the master suddenly goes down one day? www.2cto.com
Method: Connect to the slave and perform a SAVE operation. This will save a copy of the latest
dump.rdb file from the slave in its data directory. Copy this dump.rdb file to the master’s data directory.
Then restart the master.
This made me think of using a cluster solution, but Redis officially does not have a master-
master mode like mysql~ In such cases, we have to figure it out ourselves~ 
The principle of implementation~
When both Master and Slave are operating normally, the Master handles writes, and the Slave handles synchronization;
When the Master goes down and the Slave is normal, the Slave takes over the service and simultaneously disables the master-slave replication function;
And the cycle continues sequentially.
This way, if either of the two Redis servers goes down, the other will continue to provide service, without any
perceptible impact on the website, nor any data loss.
You can also implement it so that
when the Master returns to normal, it synchronizes data from the Slave, disables the master-slave replication function after synchronization, and
restores its Master identity. Meanwhile, the Slave waits for the Master to complete data synchronization, then restores its Slave identity.
Regarding high availability,
you need to separate reads and writes. For writes, point them to a VIP~ Use Keepalived with scripts for the two masters to make the judgment.
For reads, try to use HAProxy for load distribution. This way, even if one slave goes down, HAProxy will automatically remove it~
Redis installation~
wget http://redis.googlecode.com/files/redis-2.2.13.tar.gz
tar -zxf redis-2.2.13.tar.gz
cd redis-2.2.13
make
make install
Now install keepalived~
tar -xzvf keepalived-1.1.20.tar.gz
cd keepalived-1.1.20
./configure –prefix=/usr/local/webserver/keepalived
make
make install
cp /usr/local/webserver/keepalived/sbin/keepalived /usr/sbin
cp /usr/local/webserver/keepalived/etc/sysconfig/keepalived /etc/sysconfig
cp /usr/local/webserver/keepalived/etc/rc.d/init.d/keepalived /etc/init.d
mkdir /etc/keepalived
cp /usr/local/webserver/keepalived/etc/keepalived/keepalived.conf /etc/keepalived
After installation, let’s talk about the high availability part. Below is the keepalived.conf configuration file for the master node~
Use killall -0 redis-server to check if the process is alive~
You can also use the checkredis.sh script~
【Feel free to keep it simple~】 This method uses a built-in Redis utility to check if the service is alive.
#!/bin/bash
REDIS_HOME="/home/redis"
REDIS_COMMANDS="/home/redis/src" # The location of the redis binary
REDIS_MASTER_IP="172.16.0.180" # Redis MASTER ip
REDIS_MASTER_PORT="6379" # Redis MASTER port
ERROR_MSG=`${REDIS_COMMANDS}/redis-cli PING`
#
# Check the output for PONG.
#
if [ "$ERROR_MSG" != "PONG" ]
then
# redis is down, return http 503
/bin/echo -e "HTTP/1.1 503 Service Unavailable/r/n"
/bin/echo -e "Content-Type: Content-Type: text/plain/r/n"
/bin/echo -e "/r/n"
/bin/echo -e "Redis is *down*./r/n"
/bin/echo -e "/r/n"
exit 1
else
# redis is fine, return http 200
/bin/echo -e "HTTP/1.1 200 OK/r/n" br style=”color: rgb(51, 51, 51); font-family: 宋体; font-size: 14px; line-height: 28px;” /> /bin/echo -e "Content-Type: Content-Type: text/plain/r/n"
/bin/echo -e "/r/n"
/bin/echo -e "Redis is running./r/n"
/bin/echo -e "/r/n"
exit 0
fi
When in the backup state, the services running are the same as those running when it is the master~
global_defs {
router_id DBPOOL_01
}
vrrp_script chk_redis {
script "killall -0 redis-server"
interval 2
}
vrrp_instance VI_ETH0 {
interface eth0
virtual_router_id 100
nopreempt
priority 200
advert_int 1
state BACKUP
track_script {
chk_redis
}
virtual_ipaddress {
172.16.0.180
}
notify_master "/opt/redis.sh -m"
notify_backup "/opt/redis.sh -s"
notify_fault "/opt/redis.sh -k"
}
Below is the slave configuration
global_defs {
router_id DBPOOL_01
}
vrrp_script chk_redis {
script "killall -0 redis-server"
interval 2
}
vrrp_instance VI_ETH0 {
interface eth0
virtual_router_id 100
nopreempt
priority 100
advert_int 1
state BACKUP
track_script {
chk_redis
}
virtual_ipaddress {
172.16.0.180
}
notify_master "/opt/redis.sh -m"
notify_backup "/opt/redis.sh -s"
notify_fault "/opt/redis.sh -k"
}
Below is the script for /opt/redis.sh
#!/bin/sh
#
# Script to start Redis and promote to MASTER/SLAVE
# Usage Options:
# -m promote the redis-server to MASTER
# -s promote the redis-server to SLAVE
# -k start the redis-server and promote it to MASTER
#
REDIS_HOME="/home/redis"
REDIS_COMMANDS="/home/redis/src" # Directory for Redis executables
REDIS_MASTER_IP="172.16.0.180" # Redis MASTER ip
REDIS_MASTER_PORT="6379" # Redis MASTER port
REDIS_CONF="redis-mdb.conf" # Configuration file
E_INVALID_ARGS=65
E_INVALID_COMMAND=66
E_NO_SLAVES=67
E_DB_PROBLEM=68
error() {
E_CODE=$?
echo "Exiting: ERROR ${E_CODE}: $E_MSG"
exit $E_CODE
}
start_redis() {
alive=`${REDIS_COMMANDS}/redis-cli PING`
if [ "$alive" != "PONG" ]; then
${REDIS_COMMANDS}/redis-server ${REDIS_HOME}/${REDIS_CONF}
sleep 1
fi
}
start_master() {
${REDIS_COMMANDS}/redis-cli SLAVEOF no one
}
start_slave() {
${REDIS_COMMANDS}/redis-cli SLAVEOF ${REDIS_MASTER_IP} ${REDIS_MASTER_PORT}
}
usage() {
echo -e "Start Redis and promote to MASTER/SLAVE – version 0.3
(c) Alex Williams – www.alexwilliams.ca"
echo -e "/nOptions: "
echo -e "/t-m/tpromote the redis-server to MASTER"
echo -e "/t-s/tpromote the redis-server to SLAVE"
echo -e "/t-k/tstart the redis-server and promote it to MASTER"
echo -e ""
exit $E_INVALID_ARGS
}
for arg in "$@"
do
case $arg in
-m) arg_m=true;;
-s) arg_s=true;;
-k) arg_k=true;;
*) usage;;
esac
done
if [ $arg_m ]; then
echo -e "Promoting redis-server to MASTER/n"
start_redis
wait
start_master
elif [ $arg_s ]; then
echo -e "Promoting redis-server to SLAVE/n"
start_redis
wait
start_slave
elif [ $arg_k ]; then
echo -e "Starting redis-server and promoting to MASTER/n"
start_redis
wait
start_master
else
usage
fi