Persistence is to save data in memory to disk.
How to realize persistence
Make a complete backup of all data at a certain time.
For example: dump mode of MySQL and RDB mode of redis.
All the write instructions (add, delete and modify) executed by the user are backed up to the file. When restoring the data, all the backup instructions need to be executed again.
Examples: binlog of MySQL, AOF of redis, Hlog of HBase.
Introduction to AOF
As mentioned in the article “redis persistence – RDB”, if you use RDB for persistence, if redis goes down unexpectedly (such as power interruption), you may lose the data from the last backup to the period before the outage.
To avoid these problems, we can use another persistence implementation: AOF persistence.
After configuring AOF persistence, whenever redis executes a command to change the dataset, the command will be appended to the end of the AOF file. When redis is restarted, the program can re execute the commands in the AOF file to rebuild the dataset.
Three strategies of AOF persistence
Every time a new command is executed, the buffer is refreshed and the command is appended to the AOF file. Compared with other strategies, the speed is slow and the IO overhead is large, but the security is very high.
Everysec (default & recommended)
Refresh the buffer once per second to append the command to the AOF file. The speed is fast enough and only one second of data will be lost in case of failure.
Instead of refreshing the buffer, the operating system decides when to synchronize the data. The fastest, but also the least secure.
Because the operation of AOF is to add commands to the end of the file continuously, the volume of AOF file will become larger and larger with the increase of write commands. But the execution of many commands will actually cover the result of the original command (such as incr), which makes it unnecessary to keep the original command.
Therefore, redis can rebuild the AOF file without interrupting the service client. Manual execution
bgrewriteaofCommand, redis will generate a new AOF file, which contains the minimum commands needed to rebuild the current dataset.
bgrewriteaofCommands are executed asynchronously, even if
bgrewriteaofThe execution fails and no data is lost because the old AOF file is in the
bgrewriteaofIt will not be modified until it succeeds.
Redis 2.4 can automatically trigger AOF rewriting through configuration.
The role of AOF rewriting
- It can reduce the usage of disk
- Faster data recovery
#Enable AOF persistence appendonly yes #Aof persistent file name appendfilename appendonly-<port>.aof #Synchronizes buffer data to disk per second appendfsync everysec #Data persistence file storage directory dir /var/lib/redis #Do you want to unsynchronize data to AOF files when rewriting #Yes means that the data is not synchronized to the AOF file when rewriting no-appendfsync-on-rewrite yes #Minimum size to trigger rewriting of AOF file auto-aof-rewrite-min-size 64mb #File volume growth rate that triggers AOF file rewriting auto-aof-rewrite-percentage 100
To trigger AOF rewriting automatically, the following two conditions should be met simultaneously:
- aof_current_size > auto-aof-rewrite-min-size
- (aof_current_size – aof_base_size) * 100 / aof_base_size > auto-aof-rewrite-percentage
aof_ current_ Size: current size of AOF file (bytes)
aof_ base_ Size: the size of the AOF file when it was last started and rewritten (bytes)
Aof rewriting process
Advantages of AOF
- According to different fsync strategies, both performance and data security can be considered.
- Aof file is a log file that only appends. Even if the complete write command is not executed due to some reasons (disk space is full, the writing process is down, etc.), you can use redis check AOF tool to repair these problems.
- Redis can automatically perform AOF rewriting in the background when the volume of AOF file becomes too large. The rewritten new AOF file contains the minimum set of commands needed to recover the current dataset. In the process of creating a new AOF file, redis will continue to add commands to the original AOF file. Even if the rewriting process goes down, the original AOF file will not be lost. Once the new AOF file is created, redis will switch from the old AOF file to the new AOF file and start the operation of appending the new AOF file.
- Aof file orderly saves all the write operations to the database, which are saved in the format of redis command, so it is easy to analyze the content of AOF file. For example, if the flushall command is executed carelessly, as long as the AOF file is not rewritten, as long as the server is stopped, the flushall command at the end of the AOF file is removed, and redis is restarted, the dataset can be restored to the state before flushall execution.
Disadvantages of AOF
- For the same dataset, the volume of AOF file is usually larger than that of RDB file.
- According to different fsync strategies, AOF may be slower than RDB.