ASP. Net # core using sliding window to limit current and scenario analysis

Time:2022-5-2

The sliding window algorithm is used to deal with the uneven distribution of requests in the time cycle, and can deal with traffic changes more accurately. The more famous application scenario is the traffic control of TCP protocol, but today we want to talk about the application in the service flow limiting scenario.

Algorithm principle

Assuming that the service needs to limit the current 100 times per second, let’s first look at two problems of the fixed window algorithm:

Missed inspection

As shown in the figure below, the number of requests does not exceed 100 in the first and second seconds, so the current limit will not be triggered when using the fixed window algorithm. However, the number of requests in the last 500ms of the first second plus the number of requests in the first 500ms of the second second exceeds 100. At this time, it may cause damage to the system, which cannot be detected when using the fixed window algorithm.

Taigang

To solve the problem of missed detection, you may say that you can set the time window to 500ms and the current limiting threshold to 50. Let’s look at the figure below. Except that the second counting cycle exceeds 50, which triggers the current limit, the requests of the previous and subsequent counting cycles are very normal, and even will not exceed 50% of the threshold. Maybe the situation of the second counting cycle is too special, and there will be no second time in one day. If it will not affect the system, can you accommodate it! Fixed window algorithm will appear too rigid at this time.

So how can sliding window solve these two problems? Let’s look at the picture first:

As shown in the figure above:

  • The time span of the sliding window is 1 second, and the time span of each small counting cycle is 500ms. The sliding window here contains 2 small counting cycles.
  • With the advance of time, the small counting cycle contained in the sliding window will move forward in 500ms, but it always contains 2 small counting cycles.
  • When judging whether to limit current, it is necessary to add up the count values of 2 small counting cycles contained in the current sliding window.
  • Compared with the fixed window counter algorithm, the sliding window can effectively reduce missed detection. As shown in the figure above, when the sliding window moves to 500-1500ms and the total number exceeds 100, the current limit is triggered; The sliding window will not trigger current limiting at 0-1000ms and 1000-2000ms. Even if the count value of a small cycle exceeds half of the threshold, but the total number does not exceed 100, it will not limit current and can deal with rare sudden traffic.

It can also be seen from the analysis that the more small cycles of the sliding window are divided, the more accurate the detection is, but the more counts are used for tracking, and the amount of memory and calculation used will increase.

Algorithm implementation

Here are two implementation methods: in-process memory sliding window algorithm and redis based sliding window algorithm.

In process memory sliding window algorithm

Here is a high-performance method, which uses arrays to realize sliding windows. This is a special case of ring queue, as shown in the following figure:

  • Assuming that the sliding window needs 5 small counting cycles, initialize an integer array with a length of 5, and the number represents the first few elements in the array.
  • We know that the queue has a head and a tail. Take the data from the head of the queue and insert the data into the tail of the queue. The number in parentheses indicates the number of elements in the queue.
  • When the sliding window moves forward, the tail of the team moves 1 bit to the right, and the head of the team also moves 1 bit to the right.
  • Moving the tail and head of the queue to the right may overflow the array. At this time, let them return to the starting position of the array, that is, the first position of the array in the figure.

For a detailed introduction to this algorithm, you can see this article:www.jb51.net/article/200672.htm

Redis based sliding window algorithm

Based on redis, you can also use a method similar to ring queue, such as defining 5 kV as the 5 elements of the array. However, I used a more intuitive way in my previous implementation. Each small counting cycle creates a kV and sets an expiration time that absolutely exceeds the time span of the sliding window. The unused small counting cycle will not occupy memory all the time; When judging whether to trigger current limiting, it is OK to add up the count values of these small sliding windows. Of course, some details need to be improved in the actual implementation, such as how to find these small counting cycles. There will be a variety of schemes, such as saving or temporary calculation.

These operation logic can be encapsulated in a Lua script. Because Lua script is also an atomic operation when executed in redis, the current limit count of redis is naturally accurate during distributed deployment.

Application algorithm

Here, the current limiting component fireflysoft Take ratelimit as an example to realize ASP Net core.

1. Install nuget package

There are many installation methods. Just choose what you like.

Package manager command:

Install-Package FireflySoft.RateLimit.AspNetCore

Or Net command:

dotnet add package FireflySoft.RateLimit.AspNetCore

Or add the project file directly:


<ItemGroup>
<PackageReference Include="FireflySoft.RateLimit.AspNetCore" Version="2.*" />
</ItemGroup>

2. Using middleware

Using middleware in startup, the demonstration code is as follows (detailed description will be given below):

public void ConfigureServices(IServiceCollection services)
        {
           ...
           app.AddRateLimit(new InProcessSlidingWindowAlgorithm(
                new[] {
                		//The constructor has two parameters: the time length of the sliding window and the time length of the small count cycle
                    new SlidingWindowRule(TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(1))
                    {
                        ExtractTarget = context =>
                        {
                        		//Extract current limiting target
                            return (context as HttpContext).Request.Path.Value;
                        },
                        CheckRuleMatching = context =>
                        {
                        		//Judge whether the current request needs current limiting processing
                            return true;
                        },
                        Name="sliding window limit rule",
                        Limitnumber = 100, // current limit threshold, i.e. 100 requests at most in 5 seconds
                    }
                })
            );
            ...
        }

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            ...
            app.UseRateLimit();
            ...
        }

As above, you need to register the service first, and then use the middleware.

When registering a service, you need to provide a current limiting algorithm and corresponding rules:

  • Here, the in-process fixed window algorithm inprocessslidingwindowalgorithm and redisslidingwindowalgorithm can be used. A redis connection needs to be passed in. Both algorithms support synchronous and asynchronous methods.
  • The current limiting threshold is 100, the current limiting sliding window is 5 seconds, and the small counting cycle is 1 second.
  • Extracttarget is used to extract the current limiting target. Here is each different request path. If there is an IO request, the corresponding asynchronous method extracttargetasync is also supported here.
  • Checkrulematching is used to verify whether the current request is flow limited. If there is an IO request, the corresponding asynchronous method checkrulematchingasync is also supported here.
  • By default, httpstatuscode 429 will be returned when the current is limited. You can customize this value with the optional parameter error when addratelimit, as well as the contents in the HTTP header and body.

The basic use is the above example.

If it is still based on tradition Net framework, you need to_ A message processor ratelimithandler is registered in start. The algorithm and rules are shared. For details, see the instructions on GitHub:https://github.com/bosima/FireflySoft.RateLimit

FireflySoft. Ratelimit is based on Net standard’s current limiting class library has a simple and lightweight kernel and can flexibly respond to current limiting scenarios with various requirements.

Its main features include:

  • A variety of current limiting algorithms: built-in fixed window, sliding window, leaky bucket and token bucket, which can also be customized and extended.
  • Multiple count storage: at present, memory and redis storage methods are supported.
  • Distributed friendly: support unified counting of distributed programs through redis storage.
  • Flexible current limit target: various data can be extracted from the request to set the current limit target.
  • Support flow restriction penalty: the client can be locked for a period of time after triggering flow restriction and will not be allowed to access.
  • Dynamic change rules: support dynamic change of flow restriction rules during program running.
  • Custom error: you can customize the error code and error message after triggering current limiting.
  • Universality: in principle, it can meet any scene requiring current limitation.

GitHub open source address:https://github.com/bosima/FireflySoft.RateLimit

This is about ASP Net # core using sliding window current limiting is introduced here. More related ASP Net # core limited content, please search the previous articles of developeppaer or continue to browse the relevant articles below. I hope you can support developeppaer in the future!