Docker + keepalived + nginx build high availability cluster

Time:2020-10-6

Using docker + keepalived + nginx to build high availability


1. In the production environment, nginx can be used as a load balancing server. Then, in the case of a single node, nginx or this server may fail. In order to ensure high availability, the first thing to think about is to deploy another nginx server. But the IP addresses of two nginx servers are different. So how to ensure that when one server fails, the other server will be used automatically? The solution is to use keepalived.

Keepalived is a service software based on VRRP protocol (virtual routing Redundancy Protocol). VRRP protocol is to form a router group of N routers that provide the same function. In this group, there is a master and multiple backup. There is a VIP (virtual IP) that provides external services on the master. The master will send multicast. When the backup fails to receive VRRP packets, the master is considered to be down, In this case, you need to select a backup as the master according to the priority of VRRP. In this way, the service can be switched to other devices in time.

Docker + keepalived + nginx build high availability cluster

2. I use docker environment to simulate multi server environment.

docker run -d --name web1 --privileged=true centos /usr/sbin/init
docker exec -it web1 /bin/bash

Tip: because I can’t bind VIP when I run docker, I add these two parameters

  • –Privileged = true: you can really have root permission in docker
  • /Usr / SBIN / init: you can use systemctl command in docker

According to the upper operation, start three containers: web1, web2 and WebTest. Two are used to build high availability, and one is used for client connection test.

Docker + keepalived + nginx build high availability cluster

3. Enter web1 and web2 containers respectively and execute the following steps:

yum install -y nginx keepalived vim net-tools  initscripts
  • VIM: the CentOS container launched by docker does not have VIM, so you need to install it yourself
  • Net tools: you can use the ifconfig command to view the network card
  • Initscripts: you can use the service command

4. Modify nginx default index.html File, the purpose is to distinguish which nginx is currently accessed. One can be marked as master and the other as backup.

cd /usr/share/nginx/html
vim index.html

Docker + keepalived + nginx build high availability cluster

Docker + keepalived + nginx build high availability cluster

5. Create check in the / etc / keepalived directory of the two containers_ nginx.sh Documents. And add the execution permission: Chmod + X check_ nginx.sh (it is not needed here, because it is under the root permission, so it is pasted to be on the safe side). check_ nginx.sh The script file is used to check whether niginx is started.

cd /etc/keepalived
touch check_nginx.sh
chmod +x check_nginx.sh
vim check_nginx.sh

check_ nginx.sh Script file content:

#!/bin/bash
A=`ps -C nginx --no-header | wc -l`
if [ $A -eq 0 ];then
    /usr/sbin/nginx
    sleep 2
    if [ `ps -C nginx --no-header | wc -l` -eq 0 ];then
        killall keepalived
    fi
fi

6. Configure in two containers respectively keepalived.conf Documents. The file is in the / etc / keepalived directory. The configuration contents are as follows:

Configuration in master:

global_defs {
   router_ id LVS_ Level ා node name, generally use the default
}

vrrp_script chk_nginx {
    script "/etc/keepalived/check_ nginx.sh "ා detect the path of nginx script
    Interval 2 ා interval between running scripts
    Weight - 20 the weight of the change
}

vrrp_instance VI_1 {
    State master ා this node is a master. If this node is down and rebooted, it will preempt the primary node immediately
    #mcast_ src_ IP 172.17.0.2 native IP. If it is not declared, the native IP will be bound by default
    Interface eth0 ා bound network card, you can view the network card information through ifconfig
    virtual_ router_ ID 70 ා virtual route ID, which must be the same in the same group
    Priority 100 ා weight
    Nopreempt ා is defined as non preemptive, because multiple master-slave handoffs are not good for servers with large number of requests
    advert_ Int 1 ා interval between sending heartbeat and other nodes in the group: S
    virtual_ipaddress {
        172.17.0.200
    }Bound virtual IP group
    track_script {
        chk_nginx
    }# execute the previously declared script
}

Configuration in backup:

global_defs {
   router_ id LVS_ Level ා node name, generally use the default
}

vrrp_script chk_nginx {
    script "/etc/keepalived/check_ nginx.sh "ා detect the path of nginx script
    Interval 2 ා interval between running scripts
    Weight - 20 the weight of the change
}

vrrp_instance VI_1 {
    State backup ා this node is backup
    # mcast_ src_ IP 172.17.0.3 native IP. If it is not declared, the native IP will be bound by default
    Interface eth0 ා bound network card, you can view the network card information through ifconfig
    virtual_ router_ ID 70 ා virtual route ID, which must be the same in the same group
    Priority 90 ා weight, the weight of the standby node is smaller than that of the primary node
    advert_ Int 1 ා interval between sending heartbeat and other nodes in the group: s. The interval within the group must be the same
    virtual_ipaddress {
        172.17.0.200
    }Bound virtual IP group
    track_script {
        chk_nginx
    }# execute the previously declared script
}

Tip: the interface items need to be checked by ifconfig, and mine are eth0. (delete the comment part by yourself, ha ha ha ha)

7. Start the test

Start nginx in the two containers respectively

nginx

Start keepalived in the two containers respectively

systemctl start keepalived.service

Check whether VIP is bound in master. You can find that 172.17.0.200 has been bound

ip a

Docker + keepalived + nginx build high availability cluster

At this time, the third docker, WebTest, is used for testing

curl 172.17.0.200

Docker + keepalived + nginx build high availability cluster

It can be found that the current access is master.

In this case, close keepalived in web1, that is, close the master node, and then in WebTest, curl:

systemctl stop keepalived.service
curl 172.17.0.200

Docker + keepalived + nginx build high availability cluster

OK! It’s done!