Sunflower classic! A paper on nginx current limiting configuration

Time:2021-3-4

Sunflower classic! A paper on nginx current limiting configuration

_ Source: https://www.cnblogs.com/bigli…
Author: biglittlean_

1. Current limiting algorithm

Token Bucket

Sunflower classic! A paper on nginx current limiting configuration

Token Bucket

The algorithm is as follows

  • The token is generated at a fixed rate and cached in the token bucket;
  • When the token bucket is full, the redundant token is discarded;
  • The request needs to consume an equal proportion of tokens to be processed;
  • When the token is insufficient, the request is cached.

Leaky bucket algorithm

Sunflower classic! A paper on nginx current limiting configuration

Leaky bucket algorithm

The algorithm is as follows

  • Water (request) is poured into the bucket from above and discharged from below (to be processed);
  • The water that can’t flow out in time is stored in the bucket (buffer) and flows out at a fixed rate;
  • When the bucket is full, the water overflows (discards).
  • The core of this algorithm is: caching requests, uniform processing, and discarding redundant requests directly.  
    Compared with the leaky bucket algorithm, the token bucket algorithm is different in that it has not only a bucket but also a queue. The bucket is used to store the token, and the queue is used to store the request.

In terms of function, the most obvious difference between leaky bucket and token bucket algorithm is whether to allow burst processing. Leaky bucket algorithm can forcibly limit the real-time transmission (processing) rate of data without additional processing of burst traffic; while token bucket algorithm can limit the average transmission rate of data while allowing some degree of burst transmission.

Nginx speed limit module uses the leaky bucket algorithm, which can forcibly ensure that the real-time processing speed of the request will not exceed the set threshold.

The official version of nginx restricts IP connection and concurrency. There are two modules respectively

  • limit_req_zoneThe leaky bucket algorithm is used to limit the number of requests per unit time, that is, the speed limit.
  • limit_req_connIt is used to limit the number of connections at the same time, that is, concurrency limit.

limit_ req_ Zone parameter configuration

    Syntax:    limit_req zone=name [burst=number] [nodelay];    
    Default:    —    
    Context:    http, server, location

limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;

  • The first parameter is $binary_ remote_ Addr means through remote_ Addr is used as a restriction, “binary” The purpose of is to abbreviate the amount of memory occupied, is to limit the same client IP address.
  • The second parameter: zone= one:10m It means to generate a memory area with the size of 10m and the name of one to store the access frequency information.
  • The third parameter: rate = 1R / s, which means the access frequency of clients with the same identity is allowed. Here, the limit is once per second, and there can be, for example, 30R / m.

limit_req zone=one burst=5 nodelay;

  • The first parameter, zone = one, sets which configuration area to use for restriction, similar to the above limit_ req_ The name in the zone corresponds to.
  • The second parameter: burst = 5. Focus on this configuration. Burst means to set a buffer of size 5. When a large number of requests (burst) come, the requests exceeding the access frequency limit can be put into this buffer first.
  • The third parameter: nodelay. If it is set, 503 will be returned when the access frequency is exceeded and the buffer is full. If it is not set, all requests will be queued.

example:

    http {        
    limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
    server {            
    location /search/ {            
    limit_req zone=one burst=5 nodelay;       
    }   
    }        

The following configuration can restrict the access of specific UA (such as search engine)

limit_req_zone $anti_spider zone=one:10m rate=10r/s;  
 limit_req zone=one burst=100 nodelay;  
 if ($http_user_agent ~* "googlebot|bingbot|Feedfetcher-Google") {  
 set $anti_spider $http_user_agent;  
 }

Other parameters

 Syntax:    limit_req_log_level info | notice | warn | error;  
    Default:      
    limit_req_log_level error;  
    Context:    http, server, location

When the server is limited or cached due to limit, the configuration writes to the log. Delayed records are one level lower than rejected records. example:limit_req_log_level notice
The basic reason for delay is info.

    Syntax:    limit_req_status code;    
    Default:        limit_req_status 503;    
    Context:    http, server, location

Sets the return value of the reject request. The value can only be set between 400 and 599.

ngx_ http_ limit_ conn_ Module parameter configuration

This module is used to limit the number of requests for a single IP. Not all connections are counted. Connections are counted only if the server has processed the request and read the entire request header.

    Syntax:    limit_conn zone number;    
    Default:    —    
    Context:    http, server, location
    limit_conn_zone $binary_remote_addr zone=addr:10m;  
    server {       
    location /download/ {  
    limit_conn addr 1;    
    }

Only one connection per IP address is allowed at a time.

    limit_conn_zone $binary_remote_addr zone=perip:10m;  
    limit_conn_zone $server_name zone=perserver:10m;  
    
    server {      
    ...      
    limit_conn perip 10;      
    limit_conn perserver 100;   
    }

Multiple limits can be configured_ Conn instruction. For example, the above configuration will limit the number of IP connections to servers per client and the total number of connections to virtual servers.

    Syntax:    limit_conn_zone key zone=name:size;  
    Default:    —   
    Context:    http
    limit_conn_zone $binary_remote_addr zone=addr:10m;

Here, the client IP address is the key. Please note, No$ remote_addrInstead of using$ binary_remote_addrVariable.$ remote_addr
Variables can range in size from 7 to 15 bytes. The stored state occupies 32 or 64 bytes of memory on 32-bit platforms, and always occupies 64 bytes on 64 bit platforms. For IPv4 addresses,$ binary_remote_addr
The size of the variable is always 4 bytes and 16 bytes for IPv6 addresses. The storage state always occupies 32 or 64 bytes on 32-bit platform and 64 bytes on 64 bit platform. A megabyte region can hold about 32000 32 byte states or about 16000 64 byte states. If the area storage runs out, the server returns the error to all other requests.

    Syntax:    limit_conn_log_level info | notice | warn | error;  
    Default:        limit_conn_log_level error;   
    Context:    http, server, location

Set the required logging level when the server limits the number of connections.

    Syntax:    limit_conn_status code;   
    Default:        limit_conn_status 503;    
    Context:    http, server, location

Sets the return value of the reject request.

2. Actual combat

Instance 1 limit access rate

    limit_req_zone $binary_remote_addr zone=mylimit:10m rate=2r/s;    
    server {        
    location / {         
    limit_req zone=mylimit;   
    }  
    }

The above rule limits the speed of each IP access to 2R / s, and applies the rule to the root directory. What happens if a single IP sends multiple requests in a very short time?

Sunflower classic! A paper on nginx current limiting configuration

Single IP sends 6 requests in 10ms

We used a single IP to send six requests within 10 ms, only one succeeded, and the remaining five were rejected. The speed we set is 2R / s. why is only one success? Is the nginx limit wrong? Of course not, because the current limiting statistics of nginx is based on milliseconds. The speed we set is 2R / s. After a conversion, only one request is allowed for a single IP within 500ms, and the second request is allowed from 501ms.

Case 2 burst cache processing

We can see that we have sent a large number of requests in a short time, and nginx counts them according to the millisecond level accuracy, and the requests exceeding the limit are rejected directly. This is too harsh in the actual scene. In the real network environment, requests do not come at a constant speed, and there may be “sudden” requests, that is, “one by one”. Nginx takes this situation into consideration, and can enable the caching of burst requests through the burst keyword instead of directly rejecting them.  
Let’s look at our configuration:

    limit_req_zone $binary_remote_addr zone=mylimit:10m rate=2r/s;   
    server {       
    location / {       
    limit_req zone=mylimit burst=4;    
    }   
    }

We add burst = 4, which means that each key (here, each IP) allows up to four burst requests. What happens if a single IP sends six requests in 10 ms?

Sunflower classic! A paper on nginx current limiting configuration

Set burst

Compared with example 1, the number of successful cases is increased by 4, and the number of burst we set is consistent. The specific process is: one request is processed immediately, four requests are put into the burst queue, and the other request is rejected. Through the burst parameter, we make the nginx current limiter have the ability to handle burst traffic.

But please note: the function of burst is to let the redundant requests be put into the queue first and processed slowly. If the nodelay parameter is not added, the requests in the queue will not be processed immediately. Instead, they will be processed slowly according to the speed set by the rate and at a precise speed of milliseconds.

Instance 3 nodelay reduces queuing time

In example 2, we can see that by setting the burst parameter, we can allow the nginx cache to handle a certain degree of burst, and the redundant requests can be put into the queue first and processed slowly, which plays a role in smoothing the traffic. However, if the queue is set larger, the request queue time will be longer. From the user’s point of view, RT becomes longer, which is very unfriendly to users. What’s the solution? The nodelay parameter allows requests to be processed immediately when they are queued. That is to say, as long as the request can enter the burst queue, it will be processed immediately by the background worker. Please note that this means that when nodelay is set in burst, the instantaneous QPS of the system may exceed the threshold set by rate. The nodelay parameter works only when it is used with burst.

Continuing with the configuration of instance 2, we add the nodelay option:

    limit_req_zone $binary_remote_addr zone=mylimit:10m rate=2r/s;    
    server {        
    location / {    
    limit_req zone=mylimit burst=4 nodelay;   
    } 
    }

A single IP sends 6 requests in 10ms, and the results are as follows:

Sunflower classic! A paper on nginx current limiting configuration

Setting burst and nodela

Compared with example 2, the success rate of the request has not changed, but the overall time consumption has become shorter. How do you explain that? In instance 2, four requests are put into the burst queue, and the work process takes one request every 500ms (rate = 2R / s) for processing, and the last request will be queued for 2S before being processed; in instance 3, the request put into the queue is the same as that in instance 2, but the difference is that the requests in the queue have the qualification to be processed at the same time, so the five requests in instance 3 can be said to be processed at the same time It takes less time to start processing.

However, please note that although burst and nodelay can reduce the processing time of burst requests, it will not increase the upper limit of throughput in the long run. The upper limit of throughput in the long run is determined by rate, because nodelay can only guarantee that burst requests are processed immediately, but nginx will limit the speed of queue element release, just as it limits the speed of token generation in token bucket.

When you see here, you may ask, which bucket is the speed limit algorithm after adding the nodelay parameter, the leaky bucket algorithm or the token bucket algorithm? Of course, it’s the leaky bucket algorithm. Consider a case where the token of the token bucket algorithm is exhausted? Because it has a request queue, it will cache the next requests. The cache size is limited by the queue size. But does it make sense to cache these requests at this point? If the server is overloaded, the cache queue is getting longer and longer, and the RT is getting higher and higher, even if the request is processed after a long time, it is of little value to the user. So when the token is not enough, the wisest way is to directly reject the user’s request, which becomes the leaky bucket algorithm.

Example 4 custom return value

    limit_req_zone $binary_remote_addr zone=mylimit:10m rate=2r/s;  
    server {        
    location / {      
    limit_req zone=mylimit burst=4 nodelay;       
    limit_req_status 598;    
    }   
    }

By default, the status of the status return value is not configured:

Sunflower classic! A paper on nginx current limiting configuration

Status is not configured

Status of custom status return value:

Sunflower classic! A paper on nginx current limiting configuration

Custom return value

If there are errors or other problems, welcome to comment and correct. If you have any help, please click like + forward to share.

Welcome to the official account of the brother of migrant workers:The way of migrant workers’ Technology
Sunflower classic! A paper on nginx current limiting configuration