Rehash mechanism of [redis] hash table

Time:2021-4-23

The complete structure of hash table, because it is multiple hashes nested layer by layer, so it will be such a structure

 

 

 

When to trigger rehash

The process of dictionary type capacity change is called rehash, which needs to meet certain conditions to trigger the expansion mechanism
The server does not execute bgwriteaof or bgsave command at present, and the current number of key value pairs exceeds the size of one-dimensional array.

If the number of current key value pairs exceeds five times the size of one-dimensional array, the capacity will be forcibly expanded no matter whether the bgwriteaof or bgsave command is in progress.
After the expansion of hash type, the length of the array is twice of the original

Capacity reduction mechanism: if the number of current key value pairs is less than one tenth of the size of one-dimensional array, the capacity reduction process will be triggered. Capacity reduction does not consider whether the current server is carrying out bgwriteaof or bgsave commands

 

The process of progressive rehash

Using two hash tables, it is a bit similar to the migration of database. When reading, read the old database first, but not read the new database. When writing, only write the new database; Other old data are moved to the new database bit by bit

 

When the expansion is triggered, redis will first allocate a memory space for HT [1]. If the current dictionary is a relatively large dictionary, the time complexity of the whole expansion process is O (n). The direct and complete expansion mechanism may cause redis to stop serving for a period of time. In order to avoid stopping the service, the design team of redis adopts the strategy of progressive rehash. Each time, only a small part of the original hash table is relocated. In this way, all the key value pairs are migrated to the new hash table.

 

First of all, for the key query, we need to search the original hash table. If we find the corresponding value, we can return it directly. If it is not found, there are only two possibilities. One is that the key value pair has been moved to a new hash table. The other is that the key value pair does not exist at all. No matter what the possibility is, we need to go to the new hash table to find it. If it is found, we will return it. If it is not found, it means that the key value pair does not exist.

 

The steps are as follows:
1. Allocate space for the spare hash table of the dictionary
If the expansion operation is performed, the size of the spare hash table is 2n (the nth power of 2) greater than or equal to (the number of used nodes) * 2
If you are performing a shrink operation, the size of the spare hash table is the first 2n greater than or equal to the number of used nodes
2. Maintain an index counter variable rehashidx in the dictionary, and set its value to 0, indicating that the rehash work is officially started (when it is - 1, it means that no rehash is performed).
3. During rehash, every time the dictionary is added, deleted, searched or updated, in addition to the specified operation, the program will rehash all the key value pairs of HT [0] hash table in rehashidx index to HT [1]. After one rehash operation, the program will rehashidx attribute value + 1. At the same time, the rehash correlation function is invoked in serverCron, and rehash is processed in the time of 1ms, and only a small number of transfer tasks (100 elements) are handled at a time.
With the continuous execution of dictionary operation, at a certain time point, all key value pairs of HT [0] will be rehash to HT [1]. At this time, the program sets the value of rehashidx attribute to - 1, indicating that the rehash operation has been completed.

Initialize HT [1] 0-7, HT [0] is old with data 0-3

 

Start to move, move the data on HT [0] 0 to the part 4 of new HT [1]

 

 

Rehash is finished, and all of HT [0] is moved to HT [1]

 

Recommended Today

Deeply analyze the principle and practice of RSA key

1、 Preface After experiencing many dark moments in life, when you read this article, you will regret and even be angry: why didn’t you write this article earlier?! Your darkest moments include: 1. Your project needs to be connected with the bank, and the other party needs you to provide an encryption certificate. You have […]