[thinking] redis’s key deletion strategy and memory elimination mechanism

Time:2021-10-19

1. Expiration policy of redis

1. Scheduled deletion

While setting the expiration time of a key, create a timer for the key to delete the key when the expiration time of the key comes
advantage:
Ensure that the memory is released as soon as possible
Disadvantages:
1) If there are many expired keys, deleting these keys will take a lot of CPU time. When the CPU time is tight, the CPU can’t spend all its time doing important things. It also needs to take time to delete these keys.
2) The creation of timers takes time. If a timer is created for each key that sets the expiration time (a large number of timers will be generated), the performance will be seriously affected

2. Inert deletion

The key is not deleted when it expires. Check whether it expires every time you get the key from the database. If it expires, delete it and return null.
advantage:
The deletion operation only occurs when the key is fetched from the database, and only the current key is deleted. Therefore, the CPU time is relatively small, and the deletion has reached the point of necessity
Disadvantages:
If a large number of keys have not been obtained for a long time after the timeout, memory leakage may occur (useless garbage takes up a lot of memory)

3. Delete regularly

If no key in the current library has an expiration time set, directly perform the traversal of the next library, randomly obtain a key with an expiration time set, check whether the key has expired, if so, delete the key, judge whether the periodic deletion operation has reached the specified time, and if so, exit the periodic deletion directly. (20 keys are detected for each library by default)
advantage:
1) By limiting the duration and frequency of deletion operations, the CPU time occupied by deletion operations is reduced — the disadvantage of dealing with “scheduled deletion”
2) Regularly delete expired keys — deal with the disadvantage of “lazy deletion”
Disadvantages:
1) In terms of memory friendliness, it is not as good as “scheduled deletion”
2) In terms of CPU time friendliness, it is not as good as “lazy deletion”

Redis adopts the strategy of regular deletion + lazy deletion

2. Memory elimination mechanism

Redis has an expiration policy. If your redis can only store 1g of data, you write 2G in one request, and you don’t request the key in time, the lazy deletion will not take effect, and redis will occupy more and more memory.

Redis can set the memory size:

# maxmemory <bytes>
#Set the maximum memory size of redis to 100
maxmemory 100mb

If this memory size is exceeded, it will be triggeredMemory elimination mechanism

Redis has a defaultMemory elimination mechanism

# maxmemory-policy noeviction

maxmemory-policyThere are 8 values in total. When the memory is insufficient:

1) Noeviction: directly return the error message without deleting.
2) Allkeys LRU: remove the longest unused (least frequently used) keys.This is recommended.
3) Volatile LRU: among the keys with expiration time set, remove the longest unused key.
4) All keys random: randomly remove a key.
5) Volatile random: randomly remove a key from the keys with expiration time set.
6) Volatile TTL: among the keys with expiration time set, remove the keys ready to expire.
7) All keys LFU: remove the least recently used key.
8) Volatile LFU: among the keys with expiration time set, remove the least recently used key.

Differences between LRU and LFU:

LRU is the least recently used page replacement algorithm, that is, first eliminate the pages that have not been used for the longest time!

For example, there are data 1, 1, 1, 2, 2, 3
At this time, there are (1, 2) in the cache
When 3 join, you have to eliminate the previous 1 and turn it into (3, 2)

LFU is the least frequently used page replacement algorithm recently, that is, eliminate the pages that are accessed the least in a certain period of time!

For example, there are data 1, 1, 1, 2, 2, 3
In cache (1 (3 times), 2 (2 times))
When 3 is added, the latter 2 must be eliminated and become (1 (3 times), 3 (1 time))

Extension:

If my key is not set to expire, i.e. no expiration time is set. Then volatile LRU, volatile random and volatile TTL cannot be executed, just like noeviction.