We all know that redis is an in memory database, and all data is stored in memory. Once the redis process on the server exits, the data in the database will be lost.
What does persistence do? The simple understanding of persistence is to make a backup of the data in memory.
There are two methods of redis persistence, RDB persistence and AOF persistence. This article will be divided into two parts to introduce these two persistence methods and their implementation
1、 RDB persistence
Redis data persistence is to save the data in memory to disk to avoid accidental data loss. RDB persistence generates an RDB file, which is a compressed binary file. Through this file, the data in redis database can be restored. The persistence of RDB can be performed manually or automatically on a regular basis according to the server configuration items.
Now let’s learn how to create RDB files. There are two commands to create RDB files, one is save, the other is bgsave.
Executing the Save command will block the redis server process until the RDB file is created. During the blocking period, the server cannot process any command requests.
Executing bgsave command will generate a child process, and then the child process is responsible for creating RDB file, and the server parent process continues to process command requests.
The underlying functions called by the Save command and bgsave command are the same function rdbsave, except that the Save command calls this function directly, and bgsave will fork() to call this function. The pseudo code is as follows:
The loading of RDB files is automatically executed when the server starts. As long as the redis server detects the existence of RDB files at startup, the RDB files will be loaded automatically. It is worth mentioning that during the loading of RDB files, the redis server will be blocked until the loading is completed.
The bgsave command is executed without blocking the server process, so redis allows users to set the Save option of the server to make the server automatically execute the bgsave command every other period of time.
If one of the above conditions is met, bgsave command will be executed
- The first one means that the server will execute bgsave command if it has modified the database at least once in 900 seconds
- The second means that the server will execute bgsave command after it has modified the database at least 10 times in 300 seconds
- The third meaning is: the server will execute bgsave command after performing at least 10000 modifications to the database in 60 seconds
2、 Aof persistence
In addition to RDB persistence, redis also provides AOF persistence function, and their implementation methods are very different. RDB persistence records database state by saving key value pairs in database, while AOF persistence records database state by saving write commands executed by redis server.
How is AOF persistence implemented? There are three steps in AOF persistence: Command appending, file writing and file synchronization.
- Command append: when the AOF persistence function is turned on, the server will append the executed write command to the AOF buffer in a certain format after executing a write command. Aof buffer is an attribute of SDS structure maintained by redisserver structure.
- File writing: file writing refers to writing from the AOF buffer of redis to the memory buffer of the operating system. This process is to improve the efficiency of file writing, but the risk is that when the server fails, the data in the memory buffer will be lost.
- File synchronization: this process is to write the data in the memory buffer to the AOF file in the hard disk
In redis, RDB persistence is performed by default. How to turn on AOF persistence? Let’s take a look at the configuration items of AOF
- Appendonly: this parameter is the switch of AOF. If it is configured to yes, AOF persistence mechanism can be turned on. After opening AOF mechanism
- Appendfsync: we know that there is an event loop in redis. In each event loop, redis writes the contents of AOF buffer to the memory buffer of the operating system. This parameter is used to configure the update frequency of synchronizing the data in the memory buffer to the AOF file on the hard disk. There are three configuration values: always, everysec and No. Always means that each write operation will immediately synchronize the contents of the memory buffer to the AOF file on the disk. This configuration has poor performance, but can ensure that the data is not lost. Everysec means to synchronize the data in the memory buffer of the operating system to the AOF file of the disk once a second. This operation is performed by a thread with high performance. No means that the operating system controls when to synchronize the data in the memory buffer to the AOF file on the hard disk. This kind of operation will lose part of the data when the server is abnormal.
Let’s talk about the restoration process of AOF files. We know that all write commands that have been executed are stored in AOF files. So the server only needs to read in and re execute the write commands saved in AOF files to restore the database contents before the server closes.
At the same time, in order to solve the problem that AOF files are getting larger and larger in the process of redis running, redis also provides the function of AOF rewriting. The principle of AOF rewriting is not specifically introduced here. We can discuss it privately if we are interested.
3、 The difference and connection between RDB and AOF, and the situation when they work at the same time
First of all, let’s summarize the differences and connections between the two methods
- RDB persistence: on by default; full backup, saving the whole database at one time; small size, fast data recovery; server exception may lose some data; save operation will block, bgsave will not block.
- Aof persistence: closed by default; incremental backup, save one command to modify the database at a time; large volume, slow data recovery; backup frequency can be set by yourself; no blocking.
When RDB and AOF start at the same time:
- If RDB is performing snapshot operation, redis will not perform AOF rewrite; if redis is performing AOF rewrite, RDB snapshot will not be performed.
- If the RDB is performing snapshot and the user executes bgrewrite of command, AOF rewrite will not be executed until the RDB snapshot is generated.
- There are RDB snapshot files and AOF log files at the same time. When redis is restarted, AOF will be used in priority for data recovery because the logs are more complete.
For those interested in PHP back-end technology and PHP architecture technology, my official group is 1023755567Click here Learn together and discuss with each other.
There is already management in the group to sort out the knowledge system (source code, learning video and other materials). You are welcome to get it for free