Using consumer + nginx upsync to realize dynamic load

Time:2021-8-2

This is from the course I learned some time ago. I actually operated it. The detailed operation and instructions are as follows.


If nginx encounters large traffic and high load, it may not always be convenient to modify the configuration file and restart, because restoring nginx and reloading the configuration will further increase the system load and may temporarily reduce the performance. Modifying configuration files one by one is also error prone and time-consuming.

At this time, you might as well try consuming + nginx upsync module to realize the dynamic load of nginx.

nginx-upsync-module

Nginx upsync module provides dynamic load balancing. It can synchronize upstreams from consumer or etcd and dynamically modify the back-end server properties (weight, max)_ Failures, down…) without reloading nginx. In this way, we can achieve smooth scaling through it without seriously affecting performance.

Using docker to install

I have built an image 13sai / nginx Lua upsync based on CentOS 7. You can start a container using the following command

docker run -itd –name=nginx-upsync -p 8008:80 -p 9501:9501 -p 9502:9502 -p 9503:9503 -p 8500:8500 13sai/nginx-lua-upsync

Of course, you can also build your own without docker. You can add nginx upsync module for referenceNginx module Lua module

Enter container configuration

docker exec -it nginx-upsync /bin/bash

cd /usr/local/nginx/conf

echo “server host.docker.internal:9501 weight=1 fail_timeout=10 max_fails=3;” >> servers.conf

vi nginx.conf

#Nginx.conf main configuration
 ...

 upstream 13sai{
    upsync 192.168.65.2:8500/v1/kv/upstreams/test-server upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off;
    upsync_dump_path /usr/local/nginx/conf/servers.conf;
    include /usr/local/nginx/conf/servers.conf;
}

server {
    listen       80;

    location / {
        proxy_pass http://13sai;
    }

    ...

}

...

Upsync syntax description

Syntax: Syntax: upsync $consumer / etcd. API. Com: $port / V1 / kV / upstreams / $upstream_ name/ [upsync_ type=consul/etcd] [upsync_ interval=second/minutes] [upsync_ timeout=second/minutes] [strong_ dependency=off/on]
Default value: none. If the parameter is omitted, the default parameter is upsync_ interval = 5s upsync_ timeout = 6m strong_ dependency = off
Description: pull upstreams from consumer / etcd

Upsync defines the operation of pulling the latest upstream information from consumer / etcd and coexisting it locally
upsync_ Timeout defines the timeout for pulling configuration from consumer / etcd
upsync_ Interval defines the interval between pulling configuration from consumer / etc
upsync_ Type defines the type of service to use for configuration
strong_ Whether the dependency is forced to depend on the configuration server when the dependency is started. If the configuration is on, the pull fails, and nginx will also fail to be enabled
upsync_ dump_ Path defines the local file path to be persisted after the configuration is pulled from the consumer / etcd. In this way, even if the consumer / etcd fails, there will also be local backup files

Note that the following file must be. The file path and name can be customized. Nginx upsync module will cache the load information to this file, otherwise an error will be reported when nginx starts.

#Servers.conf, 192.168.x.xxx is my host IP
server 192.168.x.xxx:9501 weight=20 max_fails=1 fail_timeout=5s;

Restart nginx

/usr/local/nginx/sbin/nginx -t

/usr/local/nginx/sbin/nginx -s reload

Although we haven’t started consumer yet, it has no impact. Upsync will pull and inevitably fail. Servers.conf will not be updated, and there will be information in the error log of nginx.

Start three HTTP services with swoole

//Three servers can be started, with ports 950195029503 respectively, and the output can be modified accordingly
$http = new Swoole\Http\Server("127.0.0.1", 9501);
$http->on('request', function ($request, $response) {
    $response->end("9501");
});

Consumer installation

Here, consumer is only used as a kV storage. It is also my first time to use it, so I won’t introduce it too much.

Download address

Unzip it to the directory you need, which is mainly a consumer executable file( Here I install it on my computer instead of the docker container just now)

Command to view the document:Consul introduction and quick start

Start:

nohup ./consul agent -dev &

For convenience, we also did not start the cluster. The consumption cluster is recommended for the production environment.

UI view

http://127.0.0.1:8500/

View node

./consul members

curl 127.0.0.1:8500/v1/catalog/nodes

View Kv value

curl -v http://127.0.0.1:8500/v1/kv/\?recurse

Using consumer + nginx upsync to realize dynamic load

add to

curl -X PUT -d ‘{“weight”:20,”max_fails”:2,”fail_timeout”:5}’ http://127.0.0.1:8500/v1/kv/upstreams/test…

192.168.x.xxx here is the host IP of the docker container I created.

delete

curl -X DELETE http://127.0.0.1:8500/v1/kv/upstreams/test…


We can test and view by adding and deleting http://127.0.0.1:8008/ To view the output, you can also look at the configuration file servers.conf in nginx. You will see that you operate consumer and dynamically change the upstream of nginx, so as to realize the dynamic capacity expansion of nginx.


The study of consumer and docker is not deep enough. If there are errors in the article, please correct and communicate.

Articles related to nginx:


The technical article is also published in its official account, a history loving programmer. Welcome to scan code attention. Thank you!

Using consumer + nginx upsync to realize dynamic load

This work adoptsCC agreement, reprint must indicate the author and the link to this article

Before collecting, you might as well give it a try!!!