Redis persistent storage

Time:2021-2-27

Redis persistent storage

Redis is mostly used for caching and message middleware. When it is used as caching, data reading and writing are carried out in memory. Once the memory is powered off or the host is restarted, the data in the memory will be cleared. To ensure that the data will not be lost, redis provides us with two kinds of data persistent storage methods: RDB and AOF.

RDB persistent storage

RDB persistence is to save the data generated by the current process to the hard disk named xx.rdb In the binary file of. We can also manually use the save or bgsave command to save the newly generated data to the hard disk xx.rdb Binary file.

Save command: when using the Save command, it will block the current redis server until RDB saves all the data to the server xx.rdb This command is not recommended for online environment.

Using the Save command, the RDB stored procedure is as follows:

  • 1. The user executes the Save command to trigger RDB storage.
  • 2. The main process of redis executes RDB storage, and the program is currently blocked.
  • 3. Copy xx.rdb To a temporary file.
  • 4. Write new data to xx.rdb In the temporary file.
  • 5. Will xx.rdb The original temporary file is overwritten xx.rdb Documents.
  • 6. RDB storage ends and responds to other user requests.

Bgsave command: the redis process creates a subprocess, and the RDB stored procedure is completed by the subprocess. After the subprocess is completed, it will automatically exit the subprocess. The blocking only occurs at the stage of creating the subprocess, so the time is very short, and it will not affect the main process.

Using the Save command, the RDB stored procedure is as follows:

  • 1. The user executes bgsave command to trigger RDB storage.
  • 2. The redis main process first checks whether there are child processes. If there are child processes, this operation will not be performed. If there are no child processes, create a child process.
  • 3. The sub process executes RDB storage, and the main process responds to user requests.
  • 4. Child process copy xx.rdb To a temporary file.
  • 5. The subprocess writes the new data to the xx.rdb In the temporary file.
  • 6. The child process will xx.rdb The original temporary file is overwritten xx.rdb Documents.
  • 7. The RDB storage ends and the child process exits.

Aof persistent storage

Aof (append only file): it appends each write command to the xx.aof In the file, when the device is powered off or restarted, redis server will execute xx.aof In this way, the purpose of data recovery can be realized.

The AOF persistent storage procedure is as follows:

1. All write commands are appended to AOF_ In the buffer of buf.
2. Aof sends the data to the hard disk according to the policy in the configuration file xx.aof The command is appended to the file.

Compare RDB and AOF

1. RDB saves the data to the file in the form of binary, while AOF saves the data in the form of text, so the file generated by AOF is much larger than that generated by RDB.
2. RDB is implemented by the redis server through a child process, and the RDB is saved in a full amount. Therefore, the time interval for RDB execution should not be too frequent, otherwise the performance will be affected. In this way, there will be a gap period. When power failure or system restart occurs in this gap period, data loss will occur. Aof records the written command to the xx.aof File, so it can better ensure that the data is not lost.
3. Data recovery: direct reading by redis server in RDB mode xx.rdb File to memory, AOF mode redis server read xx.aof File, by executing the commands in the file to achieve data recovery, so its data recovery speed is slower than RDB.

RDB storage instance

Because it is a demonstration, the configuration file here has been set simply. The content of my configuration file is as follows:

#Install redis
[[email protected] ~]# wget -P /opt/source/ http://download.redis.io/releases/redis-5.0.4.tar.gz
[[email protected] ~]# tar -zxvf /opt/source/redis-5.0.4.tar.gz -C /opt/
[[email protected] ~]# cd /opt/redis-5.0.4/
[ [email protected]  Redis-5.0.4] # make & & make install // the default is / usr / local / bin, so we don't need to set environment variables

#Create working directory
[ [email protected]  ~]# mkdir /opt/redis-5.0.4/redis_ Config / # the directory where configuration files are stored
[ [email protected]  ~]#MKDIR / opt / redis-5.0.4/data / # log, RDB, AOF and other data storage directory

#Custom profile
[[email protected] ~]# vim /opt/redis-5.0.4/redis_config/rdb_config.conf 
Bind 192.168.10.100 # local address
Port 9000 # binding port
Daemonize yes # running in the background
Dir / opt / redis-5.0.4/data / # redis data storage directory
dbfilename rdb_ data.rdb      #The name of the RDB file
#The format is: save, which means that at least changes of keys will be saved once after seconds
After save 900 1 # 900 seconds, if a key changes, trigger RDB
save 300 10
save 60 10000

Verify configuration

Running redis service

[[email protected] ~]# redis-server /opt/redis-5.0.4/redis_config/rdb_config.conf

If the service runs successfully, you will see the following:

Log in to the redis database and write the data.

[ [email protected]  ~]#Redis cli - H 192.168.10.100 - P 9000 # login redis
192.168.10.100:9000 > Ping # detect connection 
PONG
192.168.10.100:9000 > set name Xiaoming # add data to database
OK
192.168.10.100:9000> set age 12
OK
192.168.10.100:9000 > keys * # view the added data
1) "age"
2) "name"
192.168.10.100:9000> get name
"XiaoMing"
192.168.10.100:9000> get age
"12"
192.168.10.100:9000 > save # trigger RDB manually
OK
192.168.10.100:9000>

Using kill to end redis server

[ [email protected]  ~]#PID of kill - 9 redis server

If the implementation is successful, you will see the following:

Verification 1: go to the / opt / redis-5.0.4/data / directory to see if the RDB is generated_ data.rdb file

Verification 2: rerun the redis server service to verify whether the data is lost

Here is a brief introduction to RDB persistent storage. What we need to pay attention to about RDB persistent storage is the Save option in the configuration file.

[[email protected] ~]# vim /opt/redis-5.0.4/redis_config/rdb_config.conf 
Bind 192.168.10.100 # local address
Port 9000 # binding port
Daemonize yes # running in the background
Dir / opt / redis-5.0.4/data / # redis data storage directory
dbfilename rdb_ data.rdb      #The name of the RDB file
#The format is: save, which means that at least changes of keys will be saved once after seconds. This is the default value given by redis, and we can adjust it according to the actual situation
After save 900 1 # 900 seconds, if a key changes, trigger RDB
save 300 10
save 60 10000

Aof storage instance

My configuration file is as follows:

[[email protected] ~]# vim /opt/redis-5.0.4/redis_config/aof_config.conf
Bind 192.168.10.100 # bind IP address
Port 9001 # binding port
daemonize yes
Working directory of dir / opt / redis-5.0.4/data / # redis
appendfilename aof_ data.aof   #The name of the AOF file
Appendonly yes # enable AOF persistent storage
Appendfsync everysec # optional values always, everysec and no
#When it is set to always, fsync (fdataasync in Linux) will be called every time the command is received to write data to the hard disk xx.aof In the document,
#A block will appear until the data is fully synchronized to the hard disk xx.aof In this mode, it can ensure that the data will not be lost, but it will seriously affect the performance of the redis server and is rarely used.
#When it is set to everysec, every time the command is written, the system write is called to write the data to the system buffer. A separate thread calls fsync once a second to synchronize the data to the system buffer xx.aof In the document,
#If the system will lose data when the thread has not called fsync, it is recommended to use this mode, which is a trade-off between performance and storage.
#When it is set to no, every time the command is written, the system write is called to write the data to the system buffer, and then the operating system is responsible for writing the data in the buffer to the system buffer xx.aof In the document,
#This cycle is within 30 seconds. This mode has the best performance, but it can't guarantee that the data won't be lost, so it's not recommended.

#Aof is to xx.aof Add the write command to the file, so the file will be larger and larger, and there may be repeated commands, so it is necessary to add the write command to the file xx.aof The volume of the file is reduced by rewriting the file and removing duplicate commands.
Auto AOF rewrite min size 64MB initial minimum value of AOF file
Auto AOF rewrite percentage 100 # rewrite after the current AOF file size is twice the size of the last rewritten AOF file

Start redis service

[[email protected] ~]# redis-server /opt/redis-5.0.4/redis_config/aof_config.conf 

Login redis and write data

[ [email protected]  ~]#Redis cli - H 192.168.10.100 - P 9001 - login redis
192.168.10.100:9001 > Ping # view link status
PONG
192.168.10.100:9001 > keys * # view keys in database
(empty list or set)
192.168.10.100:9001 > set test HelloWorld # add data
OK
192.168.10.100:9001> set damo HaHaHa
OK
192.168.10.100:9001 > keys * # view the added data
1) "test"
2) "damo"
192.168.10.100:9001> get test
"HelloWorld"
192.168.10.100:9001> get damo
"HaHaHa"
192.168.10.100:9001>

End redis service

[ [email protected]  ~]#Kill - 9 AOF process PID # kill AOF redis service

After the execution, you will see the following:

Verification 1: in/Opt / redis-5.0.4/data/ to check whether there is AOF_ data.aof

Verification 2: log in to the database to see if the data is lost

Start redis service

[[email protected] ~]# redis-server /opt/redis-5.0.4/redis_config/aof_config.conf 

Log in to the database to view the results

 Here’s a brief introduction to redis persistent storage. There are many configuration options for AOF, but I won’t list them one by one here. Interested friends can view the configuration file and learn more about it according to the specific situation.

 

Recommended Today

Practice analysis of rust built-in trait: partialeq and EQ

Abstract:Rust uses traits in many places, from simple operator overloading to subtle features like send and sync. This article is shared from Huawei cloud community《Analysis of rust built-in trait: partialeq and EQ》Author: debugzhang Rust uses traits in many places, from simple operator overloading to subtle features like send and sync. Some traits can be automatically […]