Implementing token bucket algorithm using redis

Time:2021-12-25

In the current limiting algorithm, there is a token bucket algorithm, which can deal with short burst traffic, which is particularly useful for the situation of uneven traffic in the real environment. It will not trigger current limiting frequently and is friendly to the caller.

For example, the current limit is 10qps, which will not be exceeded in most cases, but it will occasionally reach 30qps, and then it will return to normal soon. Assuming that this sudden traffic will not affect the system stability, we can allow this instantaneous sudden traffic to a certain extent, so as to bring users a better usability experience. This is where the token bucket algorithm is used.

Principle of token bucket algorithm

As shown in the figure below, the basic principle of the algorithm is: there is a token bucket with a capacity of X, and Z tokens are put into the bucket every y unit time. If the number of tokens in the bucket exceeds x, it will be discarded. When processing the request, you need to take out the token from the token bucket first. If you get the token, continue processing; If the token is not available, the request is rejected.

008i3skNly1gx3babnba4j30du0ej74o

It can be seen that it is particularly important to set the number of X, y and Z in the token bucket algorithm. Z should be slightly larger than the number of requests per y unit time, and the system will be in this state for a long time; X is the maximum number of instantaneous requests allowed by the system, and the system should not be in this state for a long time, otherwise the current limit will be triggered frequently. At this time, it indicates that the flow exceeds the expectation, and the causes need to be investigated and corresponding measures need to be taken in time.

Redis implements token bucket algorithm

I’ve seen some token buckets implemented by programs before. The way to put tokens into the bucket is to start a thread and increase the number of tokens every y unit time, or execute this process regularly in timer. I am not satisfied with this method for two reasons: one is the waste of thread resources, and the other is the inaccurate execution time due to the scheduling problem.

Here, the method to determine the number of tokens in the token bucket is calculated. First, calculate how long it took from the last request to this request, whether it reached the time threshold of the issuing card, and then increase the number of tokens and how many tokens can be put into the bucket.

Talk is cheap!

Let’s take a look at how it is implemented in redis, because it involves multiple interactions with redis. In order to improve the throughput of current limiting processing and reduce the number of interactions between programs and redis, Lua script supported by redis is adopted. The execution of lua script is atomic, so there is no need to worry about dirty data.

Code excerpt fromFireflySoft.RateLimit, it supports not only common master-slave redis deployment, but also cluster redis, so the throughput can be improved by horizontal expansion. In order to facilitate reading, some notes are added here, which are actually not available.

--Defines the return value, which is an array, including: whether to trigger current limit (1 current limit 0 passes), and the number of tokens in the current bucket
local ret={}
ret[1]=0
--Redis cluster is divided into pieces. Keys [1] is the current limiting target
local cl_key = '{' .. KEYS[1] .. '}'

--Obtain the current setting of current limit penalty. When current limit penalty is triggered, a kV with expiration time will be written
--If there is a current limiting penalty, the result [1, - 1] is returned
local lock_key=cl_key .. '-lock'
local lock_val=redis.call('get',lock_key)
if lock_val == '1' then
    ret[1]=1
    ret[2]=-1
    return ret;
end

--Part of the code is omitted here

--Obtain [the time when the token was last dropped into the bucket]. If this drop time is not set, the token bucket does not exist. At this time:
--In one case, the token bucket is defined to be full when it is executed for the first time.
--Another situation is that the over current limiting treatment is not performed for a long time, resulting in the release of kV bearing this time,
--This expiration time will exceed the time when the token is naturally put into the bucket until the bucket is full, so the token bucket should also be full.
local last_time=redis.call('get',st_key)
if(last_time==false)
then
 --Number of tokens remaining after this execution: bucket capacity - number of tokens consumed in this execution
    bucket_amount = capacity - amount;
    --Update the number of tokens to the token bucket. At the same time, there is an expiration time. If the program is not executed for a long time, the token bucket kV will be recycled
    redis.call('set',KEYS[1],bucket_amount,'PX',key_expire_time)
    --Set [the time when the token was last put into the bucket], which will be used later to calculate the number of tokens that should be put into the bucket
    redis.call('set',st_key,start_time,'PX',key_expire_time)
    --Return value [number of tokens in current bucket]
    ret[2]=bucket_amount
    --No other treatment is required
    return ret
end

--The token bucket exists. Get the current number of tokens in the token bucket
local current_value = redis.call('get',KEYS[1])
current_value = tonumber(current_value)

--Judge whether it is time to put a new token into the bucket: current time - time of last release > = time interval of release
last_time=tonumber(last_time)
local last_time_changed=0
local past_time=current_time-last_time
if(past_time0 then
        redis.call('set',lock_key,'1','EX',lock_seconds,'NX')
    end
    ret[1]=1
    return ret
end

--Here, if the token can be deducted successfully, you need to update the token bucket
if last_time_changed==1 then
    redis.call('set',KEYS[1],bucket_amount,'PX',key_expire_time)
 --If there is a new launch, update [last launch time] to the current launch time
    redis.call('set',st_key,last_time,'PX',key_expire_time)
else
    redis.call('set',KEYS[1],bucket_amount,'PX',key_expire_time)
end
return ret

From the above code, we can see that the main processing process is:

1. Judge whether it has been punished by current limiting, return directly if it has, and enter the next step if it has not.

2. Judge whether the token bucket exists. If it does not exist, first create the token bucket, and then deduct the token to return. If it exists, go to the next step.

3. Judge whether to release the token. If not, deduct the token directly. If necessary, release the token first and then deduct the token.

4. Judge the number of tokens after deduction. If it is less than 0, the current limit is returned, and the current limit penalty is set. If it is greater than or equal to 0, proceed to the next step.

5. Update the number of tokens in the bucket to redis.

You can submit and run this Lua script in redis Library of any development language if you use Net platform, you can refer to this article:ASP. Net core using token bucket to limit current

About fireflysoft RateLimit

FireflySoft. Ratelimit is based on Net standard’s current limiting class library, its kernel is simple and lightweight, and can flexibly deal with various current limiting scenarios.

Its main features include:

  • A variety of current limiting algorithms: built-in fixed window, sliding window, leaky bucket and token bucket, which can also be customized and extended.
  • Multiple count storage: memory and redis are currently supported.
  • Distributed friendly: support unified counting of distributed programs through redis storage.
  • Flexible current limiting target: various data can be extracted from the request to set the current limiting target.
  • Support flow restriction penalty: the client can be locked for a period of time after triggering flow restriction, and its access is not allowed.
  • Dynamic change rules: support dynamic change of flow restriction rules when the program is running.
  • Custom error: you can customize the error code and error message after triggering current limiting.
  • Universality: in principle, it can meet any current limiting scenario.

GitHub open source address:https://github.com/bosima/FireflySoft.RateLimit/blob/master/README.zh-CN.md

Get more knowledge of architecture. Please pay attention to the official account number FireflySoft. Original content, please indicate the source for reprint.