Service flow limiting algorithm, strategy and where to limit flow

Time:2021-10-12

Service flow limiting algorithm, strategy and where to limit flow

1. What is service flow restriction?

With the development of microservices and distributed systems, the mutual invocation between services is becoming more and more complex.

In order to ensure the stability and high availability of their own services, certain flow limiting measures should be taken when facing requests that exceed their own service capabilities.

Just like the travel during May Day and national day, the scenic spots are full and the flow of tourists is limited. In the face of high concurrency and high traffic scenarios such as second kill, big promotion, 618, double 11, possible malicious attacks and crawlers, our services also need to limit service flow.

concept: intercept requests beyond the service processing capacity and limit the traffic accessing the service.

2. Common current limiting algorithms

2.1 counter method (fixed window)

concept: count the number of incoming requests per unit time. When the statistical value reaches the current limit threshold, the current limit will be started (such as rejection and queuing). At the end of this unit time, the calculator clears and starts counting again.

Problem: at fixed time window switching, traffic up to twice the threshold may be received.

If the set time window is 5 seconds, the current limiting threshold is 10

At the 5th second, 10 flows come in, and the flow in the window does not exceed the threshold. In the 6th second, switch to the next window. At this time, the incoming traffic is 10, and the current window traffic still does not exceed the threshold. However, at the switching thresholds of the 5th and 6th seconds, the double traffic of the current limit threshold comes in a short time. At this time, the service may be abnormal due to excessive request traffic.

Service flow limiting algorithm, strategy and where to limit flow

2.2 sliding window algorithm

concept: sliding window improves the fixed window and refines it. A time window is divided into several time panes. Each pane represents a fixed time period (such as 1 minute) and has an independent counter. After a fixed time (such as 1 minute), move the sliding window forward one grid. The finer the pane division in the sliding window, the more accurate the current limiting statistics.

If the set time window is 5 seconds, the current limiting threshold is 10

At the 5th second, 10 flows come in, and the flow in the window does not exceed the threshold. At the 6th second, the incoming flow is 10. At the same time, the sliding window moves one grid to the right. At this time, the flow in the sliding window (2nd ~ 6th seconds) is 20, which is greater than the current limiting threshold and starts current limiting.

Service flow limiting algorithm, strategy and where to limit flow

2.3 funnel algorithm

concept: request to fill the funnel like water and then flow out at a fixed rate. Before the funnel is full, the request can enter all the time; If the funnel is full, the request is rejected.

Funnel algorithm can smooth traffic, but it can not solve the problem of sudden increase of traffic.

Service flow limiting algorithm, strategy and where to limit flow

2.4 token bucket algorithm

concept: at a constant rate(Token generation rate)Put the token into the token bucket. The token bucket is full(Token bucket size)Cannot be placed. After the request arrives, the token is obtained first. After receiving the token, the request is processed and the obtained token is deleted. When the token is insufficient, the request cannot obtain the token and the request is rejected.

Token bucket algorithm can smooth flow restriction and tolerate burst traffic.

Service flow limiting algorithm, strategy and where to limit flow

3. Current limiting strategy

3.1 denial of service

When the request traffic reaches the current limit threshold, the redundant requests are directly rejected.
Requests from different sources such as designated domain name, IP, client, application and user can be rejected by design.

3.2 delay processing

Add redundant requests to the cache queue or delay queue to cope with the sudden increase of short-term traffic. After the peak period, the accumulated request traffic will be processed gradually.

3.3 request classification (priority)

Priority is set for requests from different sources, and higher priority requests are processed first. For example, VIP customers and important business applications (e.g. transaction service has priority over log service)

3.4 dynamic current limiting

It can monitor the relevant indicators of the system, evaluate the system pressure, and dynamically adjust the current limiting threshold through the registration center, configuration center, etc.

3.5 monitoring and early warning & dynamic capacity expansion

If there is an excellent service monitoring system and automatic deployment and release system, the operation of the system can be automatically monitored through the monitoring system, and early warning can be given by e-mail, SMS and other means for the sudden increase of service pressure and large write in of traffic in the short term.

Under specific conditions, relevant services can be automatically deployed and published to achieve the effect of dynamic capacity expansion.

4. Current limiting position

4.1 access layer current limiting

The domain name or IP can be restricted through nginx, API routing gateway, etc., and illegal requests can be intercepted

4.2 application of current limiting

Each service can have its own single machine or cluster current limiting measures, or call a third-party current limiting service

4.3 current limit of basic services

Database: limit database connection and read / write rate

Message queue: limit consumption rate (consumption, consumption thread)


If this article is useful to you, clickfabulousGo again!
If you have any questions, please leave a comment!
Welcome to reprint, please indicate the source!

Recommended Today

Heavy! With the release of JDK 17, Oracle announced that JDK 17 is officially free..

Previous edition:JDK 16 was officially released and 17 new features were released at one time… No! JDK 17 officially released + free It’s awesome. JDK 16 has just been released for half a year (2021 / 03 / 16), and JDK 17 arrives as scheduled (2021 / 09 / 14). At this time, it’s awesome. […]