SDS of redis data structure

Time:2021-3-6

SDS(simple dynamic string), simple dynamic string. S and it’s called hacking string. Where hack is stored is the length of the string and the remaining space in SDS. The implementation of SDS insds.cIn the middle.

C language string uses a character array of length N + 1 to represent the string of length N, and the last element of the character array is always empty character ‘\ 0’. This way of storage, there are security risks, and it can not meet the efficiency requirements.

So redis doesn’t use C nativestringIt’s built on its ownSDS. In redis, C language strings are only used in places where there is no need to modify string values, such as logs.

In redis, key value pairs containing string values are implemented by using SDS. In addition, SDS is also used for AOF buffer and client state input buffer.

Definition of SDS

struct sdshdr{
     //Byte array
     char buf[]; 
     //Number of bytes used in buf array
     int len;
     //Number of unused bytes in buf array
     int free;
}

As shown in the figure above, len indicates that the SDS stores a 6-byte long string (excluding the terminator), free indicates that the SDS still has 6 bytes of unused space, and buf is a char type stringarrayTo save the string value stored in the SDS.

Efficient

Compared with C language string, the time complexity of getting string length is reduced to 0O(1)The acquisition length of C isO(N) Traverse the entire array.

security

meanwhileSDSPrevent buffer overflow, will not cause array data insecurity like C, absolutely will not cross the boundary.

When the SDS needs to be modified, the API will first check whether the current remaining space of the SDS meets the space required after modification. If not, the API will automatically expand the space of the SDS to enough space, and then proceed to the next operation, so the SDS will not have the problem of buffer overflow.

Reduce memory allocation

The bottom layer of C language word native character string is realized by using a char type data with the length of N + 1 characters, so every time a native string is increased or shortened, the program needs to reallocate the memory of the string array once

At the same time, because memory reallocation involves complex algorithms and may need to execute system calls, it is usually a time-consuming operation. Redis is often used in the situation where the speed is strict and the data is frequently modified. If the memory reallocation is required every time the string is modified, it will have a great impact on the performance.

After allocating memory (usually there will be surplus space, that is, the pre allocation of space), SDS maintains the used and unused memory through len and free, and does not rely on the system to re partition, which can effectively improve the performance.

Space pre allocation

It is used for string growth operation. When the string grows, the program will first check whether it needs to expand the SDS space. If it needs to expand, the program will not only allocate the necessary space for modification to the SDS, but also allocate additional unused space for the SDS. The formula of the extra allocated unused space is as follows:

SDS space < 1MB

If the length of SDS (the value of len attribute after modification) is less than 1MB after modification, unused space of the same size as len attribute will be allocated, and the value of len attribute and free attribute of SDS will be the same. For example, if the len of SDS is changed to 10 bytes after modification, then the program will allocate 10 bytes of unused space, and the actual length of the buf array of SDS is changed to 10 + 10 + 1 = 21 (an extra byte is used to save the terminator)

SDS space > 1MB

If the length of SDS is greater than or equal to 1MB after modification, the program will allocate 1MB of unused space. For example, the modified len will be 10MB, then the program will allocate 1MB of unused space, and the length of SDS bug array is 10MB + 1MB + 1byte

SDS space > 512MB

Game over ~ error!

Inert space release

It is used to optimize the string shrinking operation of SDS. When the string shrinks, the program will not immediately reallocate the memory to recover the extra space after shrinking, but use the free attribute to record it for future use.

Through space pre allocation, redis can reduce the number of memory reallocation required for continuous string growth operations. Through the release of inert space, SDS avoids the memory reallocation required for shortening strings, and provides optimization for possible string growth operations in the future.

Recommended Today

Third party calls wechat payment interface

Step one: preparation 1. Wechat payment interface can only be called if the developer qualification has been authenticated on wechat open platform, so the first thing is to authenticate. It’s very simple, but wechat will charge 300 yuan for audit 2. Set payment directory Login wechat payment merchant platform( pay.weixin.qq . com) — > Product […]