Solve ASP Net # core using leaky bucket algorithm to limit current

Time:2022-4-30

Leaky bucket algorithm is one of the four mainstream algorithms for current limiting. Its application scenarios are not introduced in various materials. Generally, it is applied in network flow control. Here are two examples:

1. At present, home internet access will limit a fixed bandwidth, such as 100m, 200m, etc. there are many users in a building. How can operators ensure that some users do not use too much bandwidth, thus affecting others? At this time, the leaky bucket algorithm can be used to limit the maximum bandwidth of each user accessing the network. Of course, it will be much more complicated than this.

2. There is an ancestral interface, which was written without any protection measures at that time. Now it will crash if the traffic is a little large, but no one can change the code. At this time, the leaky bucket algorithm can also be used to encapsulate the interface, rectify the external request through the leaky bucket algorithm, and then forward it to the interface. At this time, the access frequency will not exceed the threshold, and the interface will not crash.

Algorithm principle

Having said so much, how does the leaky bucket algorithm solve the problem? Please look at the picture below.

After receiving the request, first put the request into a leaky bucket, the leaky bucket leaks out the request at a constant rate, and then the leaked request is processed; If the request is received too fast and the leaky bucket is full, the new request is discarded.

It can be seen that the leaky bucket algorithm is mainly output in the way of constant speed to provide a stable input for subsequent data processing. In this way, it can cope with a certain burst traffic, so that the system will not crash due to the sudden increase of requests. Just by increasing the delay, there will be a little waste of resources, which is different from the processing method of token bucket. For the token bucket algorithm, see this article:ASP. Net core using token bucket to limit current

Another advantage not often mentioned is that constant speed output can sometimes improve efficiency. For example, if two requests are allowed to leak at one time, the two processes can be combined into one process. If each process involves network IO, the combined process has the opportunity to reduce the overhead of network io.

Algorithm implementation

Here are two implementation methods: in-process memory leaky bucket algorithm and redis based leaky bucket algorithm.

In process memory leaky bucket algorithm

Here, the leakage quantity is calculated at the time of request. There is no separate leakage processing. The algorithm described is slightly complex, but it only needs a little patience and is easy to understand.

Let’s first define several variables:

  • The leakage rate is expressed by [y per x time period]. X time period is generally a time span of several seconds, minutes, hours, etc.
  • The start time of the current time cycle is represented by TS, the end time of the current time cycle is represented by TE, and the current time is represented by Ti.
  • The capacity of leaky bucket is expressed by Z.
  • The number of all requests in x time is expressed in n.

When the request arrives, it can be processed in the following order:

  • If Ti TS < = x, it means that it is still in the current time cycle. First increase the value of N:

Compare N and Y. if n < = y, the request does not need to wait and directly leaks out and enters the processing stage;

If n > y, compare n with y + Z:

If n < = y + Z, request to enter the leaky bucket for waiting. The waiting time is: (math. Ceiling ((n-y) / y) – 1) * x + (TE – Ti). After waiting, the leaky bucket will enter the processing stage;

If n > y + Z, the request cannot enter the leaky bucket and can only be discarded. In practice, the request is rejected;

  • If Ti TS > x, you need to create a new time period:

Several time periods have elapsed in the calculation: PN = math ceiling((Ti-Te)/X);

  • Reset the values of TS and Te: TS = last TS + PN * x, TE = TS + X;
  • Calculate the maximum leakage quantity during this period: Yo = PN * y;
  • Calculate the value of N: n = n-yo < = 0? 0: N-Yo;
  • At this time, it conforms to Ti TS < = x, and it is in the current time cycle. Then return to the above steps for processing in turn.

Redis based leaky bucket algorithm

The above algorithm can also be implemented based on redis, but the representation of variables is changed to redis kV, and the algorithm logic is the same.

These operation logic can be encapsulated in a Lua script. Because Lua script is also an atomic operation when executed in redis, the current limit count of redis is naturally accurate during distributed deployment.

Application algorithm

Although it is not commonly used in business services, fireflysoft, a current limiting component, is used here Take ratelimit as an example to realize ASP Net core leakage bucket algorithm current limiting.

1. Install nuget package

There are many installation methods. Just choose what you like.

Package manager command:

Install-Package FireflySoft.RateLimit.AspNetCore

Or Net command:

dotnet add package FireflySoft.RateLimit.AspNetCore

Or add the project file directly:


<ItemGroup>
<PackageReference Include="FireflySoft.RateLimit.AspNetCore" Version="2.*" />
</ItemGroup>

2. Using middleware

Using middleware in startup, the demonstration code is as follows (detailed description will be given below):

public void ConfigureServices(IServiceCollection services)
        {
           ...
           app.AddRateLimit(new InProcessLeakyBucketAlgorithm(
                new[] {
                		//Three parameters: the capacity of the leaky bucket, the quantity of leakage per unit time, and the unit time of leakage
                    new LeakyBucketRule(20,10, TimeSpan.FromSeconds(1))
                    {
                        ExtractTarget = context =>
                        {
                        		//Extract current limiting target
                            return (context as HttpContext).Request.Path.Value;
                        },
                        CheckRuleMatching = context =>
                        {
                        		//Judge whether the current request needs current limiting processing
                            return true;
                        },
                        Name="leaky bucket limit rule",
                    }
                })
            );
            ...
        }

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            ...
            app.UseRateLimit();
            ...
        }

As above, you need to register the service first, and then use the middleware.

When registering a service, you need to provide a current limiting algorithm and corresponding rules:

  • In process leaky bucket algorithm inprocessleakybucketalgorithm and redisleakybucketalgorithm can be used here. A redis connection needs to be passed in. Both algorithms support synchronous and asynchronous methods.
  • The capacity of the leaky bucket is 20, the quantity of leakage per unit time is 10, and the unit time of leakage is 1 second. In other words, if 10 requests leak out in one second, the processing of more than 10 requests in one second will be delayed. Plus the capacity of the leaky bucket, the flow of more than 30 requests in one second will be limited.
  • Extracttarget is used to extract the current limit target. Here is each different request path. You can extract key data from the current request according to your needs, and then set various current limit targets. If there is an IO request, the corresponding asynchronous method extracttargetasync is also supported here.
  • Checkrulematching is used to verify whether the current request is flow limited. The incoming object is also the current request, which is convenient to extract key data for verification. If there is an IO request, the corresponding asynchronous method checkrulematchingasync is also supported here.
  • By default, httpstatuscode 429 will be returned when the current is limited. You can customize this value with the optional parameter error when addratelimit, as well as the contents in the HTTP header and body.

The basic use is the above example.

If it is still based on tradition Net framework, you need to_ A message processor ratelimithandler is registered in start. The algorithm and rules are shared. For details, see the instructions on GitHub:https://github.com/bosima/FireflySoft.RateLimit

FireflySoft. Ratelimit is based on Net standard’s current limiting class library has a simple and lightweight kernel and can flexibly respond to current limiting scenarios with various requirements.

Its main features include:

  • A variety of current limiting algorithms: built-in fixed window, sliding window, leaky bucket and token bucket, which can also be customized and extended.
  • Multiple count storage: at present, memory and redis storage methods are supported.
  • Distributed friendly: support unified counting of distributed programs through redis storage.
  • Flexible current limit target: various data can be extracted from the request to set the current limit target.
  • Support flow restriction penalty: the client can be locked for a period of time after triggering flow restriction and will not be allowed to access.
  • Dynamic change rules: support dynamic change of flow restriction rules during program running.
  • Custom error: you can customize the error code and error message after triggering current limiting.
  • Universality: in principle, it can meet any scene requiring current limitation.

GitHub open source address:https://github.com/bosima/FireflySoft.RateLimit

This is about ASP Net core using leaky bucket algorithm to limit current is introduced here, more related ASP Net core streaming content, please search the previous articles of developeppaer or continue to browse the relevant articles below. I hope you can support developeppaer in the future!