Smooth upgrade and rollback of nginx version in 1 minute

Time:2021-8-14

Today, let’s talk about a situation often encountered in the actual production environment of enterprises, upgrading nginx to a new version and how to roll back to the old version.

1. Environment introduction

The two nginx versions prepared today are as follows:


[[email protected] ~]# cd /download/nginx/
[[email protected] nginx]# ll
total 1952
-rw-r--r-- 1 root root 981687 Oct 17 2017 nginx-1.12.2.tar.gz
-rw-r--r-- 1 root root 1015384 Dec 4 09:58 nginx-1.14.2.tar.gz

2. Compile and install old and new versions

Compile and install nginx-1.12.2


[[email protected] nginx]# tar zxf nginx-1.12.2.tar.gz 
[[email protected] nginx]# cd nginx-1.12.2
[[email protected] nginx-1.12.2]# ./configure --prefix=/usr/local/nginx-1.12.2
[[email protected] nginx-1.12.2]# echo $?
0
[[email protected] nginx-1.12.2]# make && make install
[[email protected] nginx-1.12.2]# echo $?
0
[[email protected] nginx-1.12.2]# ll /usr/local/nginx-1.12.2/
total 0
drwxr-xr-x 2 root root 333 Mar 1 09:01 conf
drwxr-xr-x 2 root root 40 Mar 1 09:01 html
drwxr-xr-x 2 root root  6 Mar 1 09:01 logs
drwxr-xr-x 2 root root 19 Mar 1 09:01 sbin

Compile and install nginx-1.14.2


[[email protected] ~]# cd /download/nginx/
[[email protected] nginx]# tar zxf nginx-1.14.2.tar.gz 
[[email protected] nginx]# cd nginx-1.14.2
[[email protected] nginx-1.14.2]# ./configure --prefix=/usr/local/nginx-1.14.2
[[email protected] nginx-1.14.2]# echo $?
0
[[email protected] nginx-1.14.2]# make && make install
[[email protected] nginx-1.14.2]# echo $?
0
[[email protected] nginx-1.14.2]# ls -l /usr/local/nginx-1.14.2/
total 0
drwxr-xr-x 2 root root 333 Mar 1 09:03 conf
drwxr-xr-x 2 root root 40 Mar 1 09:03 html
drwxr-xr-x 2 root root  6 Mar 1 09:03 logs
drwxr-xr-x 2 root root 19 Mar 1 09:03 sbin

Here, two versions of nginx software have been deployed.

3. Start old version nginx


[[email protected] ~]# /usr/local/nginx-1.12.2/sbin/nginx -t
nginx: the configuration file /usr/local/nginx-1.12.2/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx-1.12.2/conf/nginx.conf test is successful
[[email protected] ~]# /usr/local/nginx-1.12.2/sbin/nginx
[[email protected] ~]# ps -ef|grep nginx
root    6324   1 0 09:06 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
nobody   6325  6324 0 09:06 ?    00:00:00 nginx: worker process
root    6327  1244 0 09:06 pts/0  00:00:00 grep --color=auto nginx
[[email protected] ~]# lsof -i :80
COMMAND PID  USER  FD  TYPE DEVICE SIZE/OFF NODE NAME
nginx  6324  root  6u IPv4 26324   0t0 TCP *:http (LISTEN)
nginx  6325 nobody  6u IPv4 26324   0t0 TCP *:http (LISTEN)

4. Upgrade to the new version

Version upgrade is actually an upgrade of binary files. The process is as follows:

[[email protected] ~]# /usr/local/nginx-1.12.2/sbin/nginx -v
nginx version: nginx/1.12.2
[[email protected] ~]# cd /usr/local/nginx-1.12.2/sbin/
[[email protected] sbin]# mv nginx nginx-1.12.2
#First, back up the original old version of nginx binaries
[[email protected] sbin]# cp /usr/local/nginx-1.14.2/sbin/nginx ./
#Copy the new version of the binary to the current directory

Next, perform the smooth upgrade operation


[[email protected] ~]# ps -ef|grep nginx
root    6324   1 0 09:06 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
nobody   6325  6324 0 09:06 ?    00:00:00 nginx: worker process
root    6338  1244 0 09:11 pts/0  00:00:00 grep --color=auto nginx
[[email protected] ~]# kill -USR2 6324
[[email protected] ~]# ps -ef|grep nginx
root    6324   1 0 09:06 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
nobody   6325  6324 0 09:06 ?    00:00:00 nginx: worker process
root    6340  6324 0 09:12 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
nobody   6341  6340 0 09:12 ?    00:00:00 nginx: worker process
root    6343  1244 0 09:12 pts/0  00:00:00 grep --color=auto nginx

At this time, the new master process has been started normally, but the old work process also exists, so we use the following command to send a smooth stop signal to the old work process, as follows:


[[email protected] ~]# kill -WINCH 6324
[[email protected] ~]# ps -ef|grep nginx
root    6324   1 0 09:06 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
root    6340  6324 0 09:12 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
nobody   6341  6340 0 09:12 ?    00:00:00 nginx: worker process
root    6346  1244 0 09:14 pts/0  00:00:00 grep --color=auto nginx

At this point, the old work process has stopped. Next, let’s test whether it can be accessed normally:

It can be accessed normally. In fact, this smooth upgrade action is completely imperceptible to the accessing users, so the nginx hot deployment has been completed.


[[email protected] ~]# /usr/local/nginx-1.12.2/sbin/nginx -v
nginx version: nginx/1.14.2

The view version is also the latest version. The upgrade is complete.

Note: if there is no problem after the version upgrade and the old master process needs to be closed, you can use the following command:


kill -QUIT old_master_PID

5. Version rollback

For upgrading, the most difficult thing is not upgrading, but rollback, because the probability of rollback exists in the actual production environment. For example, the new version is incompatible with the existing application due to some unknown bugs, or the operation is unstable, etc.

Therefore, for operation and maintenance engineers, fault rollback is the focus.

In the above results, we can also see that the old master process always exists. It will not close itself before it is closed manually. This design is beneficial. The advantage is to quickly roll back to the previous stable version if there is a problem after upgrading the new version.


[[email protected] ~]# ps -ef|grep nginx
root    6324   1 0 09:06 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
root    6340  6324 0 09:12 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
nobody   6341  6340 0 09:12 ?    00:00:00 nginx: worker process
root    6350  1244 0 09:23 pts/0  00:00:00 grep --color=auto nginx
[[email protected] ~]# cd /usr/local/nginx-1.12.2/sbin/
[[email protected] sbin]# mv nginx nginx-1.14.2
[[email protected] sbin]# mv nginx-1.12.2 nginx
[[email protected] sbin]# kill -USR1 6324
[[email protected] sbin]# ps -ef|grep nginx
root    6324   1 0 09:06 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
root    6340  6324 0 09:12 ?    00:00:00 nginx: master process /usr/local/nginx-1.12.2/sbin/nginx
nobody   6341  6340 0 09:12 ?    00:00:00 nginx: worker process
root    6355  1244 0 09:24 pts/0  00:00:00 grep --color=auto nginx
[[email protected] sbin]# ./nginx -v
nginx version: nginx/1.12.2

It is found from the above results that the previous version has been rolled back smoothly. Next, test whether it can be accessed normally:

It can be accessed normally, so the rollback operation is also imperceptible to the user.

The above is the whole content of this article. I hope it will be helpful to your study, and I hope you can support developpaer.