Redis – 5. RDB & AOF of redis persistence

Time:2019-11-7

Redis persistence implementation


  • snapshot
    A full backup of data at a point in time. For example, Linux snapshot backup, redis RDB, MySQL dump.
  • Journal
    All operations of data are recorded in the log, and the log is executed again when recovery is needed. MySQL biglog、Redis AOF。
RDB

What is RDB

A snapshot of the data in redis memory is generated completely. The file ending with. RDB is saved on the hard disk. When it needs to be recovered, it is loaded into memory from the file.

Three triggering modes of RDB
  • Save command trigger (sync)
[[email protected] ~]$ redis-cli
127.0.0.1:6379> save
OK

When save is executed, redis will be blocked. All data operation commands are queued for it to complete.
File policy: a new temporary file is generated. When save is finished, replace the old one with the new one.

  • Bgsave command trigger (asynchronous)
[[email protected] ~]$ redis-cli
127.0.0.1:6379> bgsave
Background saving started

When the client issues the bgsave command to the redis server, redis will fork out a sub process to generate the RDB file. When the file is generated, the subprocess feeds back to the main process. Fork subprocesses also block, but normally fork processes are very fast.
File policy: same as Save command.

  • Profile configuration rule auto trigger
To configure seconds changes Effect
save 900 1 Change 1 piece of data in 900 seconds, and automatically generate RDB file
save 300 10 Change 10 pieces of data within 300 seconds and generate RDB file automatically
save 60 10000 Change 10000 pieces of data within 60 seconds, and automatically generate RDB files

PS: none of the three rules is recommended.

RDB automatic rule configuration

#Configure automatic generation rules. It is generally not recommended to configure automatic generation of RDB files
save 900 1
save 300 10
save 60 10000
#Specify the RDB file name
dbfilename dump-${port}.rdb
#Specify RDB file directory
dir /opt/redis/data
#Bgsave error, stop writing
stop-writes-on-bgsave-error yes
#RDB files are in compressed format
rdbcompression yes
#Check the RDB file
rdbchecksum yes
The triggering method of RDB can not be ignored

  • Full volume replication
    When the master and slave copy, the master will automatically generate the RDB file (the master and slave will synchronize the data according to the RDB file).
  • debug reload
    Redis provides a debug level restart, which can generate RDB files without emptying memory.
  • shutdown
    Closing redis will trigger the generation of RDB files.
Problems in RDB

  • Time consuming, memory consuming, IO performance consuming
    It takes time to dump all the data in memory to the hard disk. Bgsave mode fork() subprocess consumes extra memory. A large number of hard disk reads and writes consume IO performance.
  • Uncontrollable, lost data
    During downtime, the memory data written after the last snapshot will be lost.
RDB summary

  • RDB is a snapshot from redis memory to hard disk for persistence.
  • Save usually blocks redis.
  • Bgsave usually does not block redis, but it will fork new processes.
  • Save auto configuration will be executed if it meets any requirements.
  • Time consuming, memory consuming, IO performance consuming
  • Uncontrollable, lost data
AOF

What is AOF

It is to write logs. Each time you execute the redis write command, the command records logs (log files ending in. AOF) at the same time. When redis goes down, data can be recovered as long as log playback is performed.

Three strategies of AOF

First, redis executes the write command to flush the command into the hard disk buffer

  • always
    Always flush buffer files to hard disk (even if sexual).
  • Everysec (recommended)
    Refresh the buffer once per second to synchronize the hard disk data.
    Compared with always, the hard disk can be protected under the condition of high write volume. One second of data will be lost in case of failure
  • no
    Refresh policy makes the system decide (uncontrollable).
  • Comparison of three strategies
command Advantage shortcoming
always No data loss IO costs a lot, and the general SATA disk is only a few hundred TPS
everysec One second of data lost Lost a second of data
no System decision It’s uncontrollable. I don’t know when to brush the disc or how much data will be lost

The everysec policy is usually used, which is also the default policy of AOF.

AOF rewrite

Aof rewriting is to simplify the expired, useless, repeated and optimized commands. Only take the final valuable result. Although write operations are frequent, the amount of key defined by the system is relatively limited.
Aof rewriting can greatly reduce the size of the final log file. So as to reduce the disk consumption and speed up data recovery. For example, we have a counting service with many auto increment operations, such as a key auto increment to 100 million, which is 100 million incrs for AOF files. Aof rewriting only records one record.

Two ways of AOF rewriting

  • Bgrewriteaof command triggers AOF rewriting
    The redis client sends bgrewriteaof command to redis, and the redis server forks a subprocess to complete the AOF rewriting. The AOF rewrite here is to trace the data in redis memory back to the AOF file. Instead of rewriting the AOF file to generate a new AOF file to replace.
  • Aof override configuration

    • Auto AOF rewrite min size: the size required for AOF file rewriting
    • Auto AOF rewrite percentage: AOF file growth
    • Aof? Current? Size: counts the current size of AOF in bytes
    • Aof? Base? Size: the size (in bytes) of the last time AOF was started and rewritten
  • The trigger time of AOF automatic rewriting should meet the following two points at the same time:

    • aof_current_size > auto-aof-rewrite-min-size
    • aof_current_size – aof_base_size/aof_base_size > auto-aof-rewrite-percentage
Aof override configuration

#Open the append operation of normal AOF
appendonly yes
#Aof filename
appendfilename "appendonly-${port}.aof"
#Brush disk every second
appendfsync everysec
#File directory
dir /opt/redis/data
#Aof rewrite growth rate
auto-aof-rewrite-percentage 100
#Aof override minimum size
auto-aof-rewrite-min-size 64mb
#Whether to pause the append operation during AOF rewrite. Aof rewriting consumes disk performance very much, and in the normal AOF process, data will also be flushed to the disk.
#It is usually biased to consider performance and set to yes. In case of failure of rewriting, the data of normal AOF will be lost during this period, because we chose to discard the normal AOF disk during rewriting.
no-appendfsync-on-rewrite yes
RDB & AOF

RDB vs AOF

command RDB AOF Explain
boot priority low high When both RDB and AOF are enabled, select AOF for recovery after redis restarts. In most cases, it holds data that is newer than RDB
volume Small large RDB is stored in binary mode and compressed. Although AOF is rewritten by AOF, its volume is relatively large. After all, it is in the form of logging
Recovery speed fast slow RDB is small in size and fast in recovery. Large volume of AOF, slow recovery
data security Lose data Based on Strategy RDB loses the data after the last snapshot. AOF decides whether to lose the data according to the strategies of always, everysec and No
Severity heavy light Aof is an append log, so it is a lighter operation. RDB is a CPU intensive operation, which consumes a lot of memory for disk and fork
RDB best strategy

  • It is recommended to turn off RDB
    It is recommended to turn off RDB for both redis master and slave nodes. However, turning off is not absolute. RDB is still used for master-slave replication.
  • Use as data backup
    RDB is a very heavy operation, but it is very useful for data backup. The file size is relatively small, and data can be backed up by day or by hour.
  • Master slave, slave drive?
    In very different scenarios, you need to open RDB from the node, and you can save a historical RDB file like this locally. Although the slave node does not read or write, redis is often deployed in a single machine with multiple deployments. Because RDB is a very heavy operation, it will still affect the CPU, hard disk and memory. Set according to the actual needs.
Aof best strategy

  • It is recommended to turn on AOF
    If the redis data is only used as the cache of the data source, and reloading from the data source after the cache is lost will not cause too much pressure on the data source, in this case. Aof can be turned off.
  • Aof rewrite centralized management
    In the case of single machine and multi deployment, a large number of forks may be full of memory.
  • everysec
    It is recommended to use the disk swiping per second strategy
Best strategy

  • Small fragment
    Use maxmemory to plan the maximum memory of redis.
  • Caching and storage
    Decide which policy to use based on the characteristics of cache and storage
  • Monitoring (hard disk, memory, load, network)
  • Enough memory
    Don’t plan all the memory of the machine for redis. Otherwise, there will be many problems. Like client buffer, it is not limited by maxmemory. Improper planning may cause problems such as swap and oom.
Common problems of development and operation

Fork operation

Fork is a synchronous operation. When bgsave and bgrewriteaof are executed, the fork operation will be executed

  • Improving fork

    • Give priority to physical machines or other virtualization technologies that can effectively support fork operations;
    • Control the maximum available memory of redis instance maxmemory;
      The fork operation is only a copy of the memory page. In most cases, it is relatively fast. The larger the redis memory, the larger the memory page. You can use maxmemory to plan redis memory to avoid slow fork.
    • Reasonable configuration of Linux memory allocation strategy: VM. Overcommit? Memory = 1
      If there is not enough memory in fork, it will block. Vm.overcommit’u memory of Linux is 0 by default, no additional memory will be allocated
Subprocess overhead and optimization

Bgsave and bgrewriteaof will perform the fork operation to generate child processes.

  • CPU

    • Overhead: RDB and AOF file generation are CPU intensive;
    • Optimization: no CPU binding, no CPU intensive application deployment together;
  • Memory

    • Cost: fork memory cost
    • Optimization: echo never > / sys / kernel / mm / transparent hugepage / enabled
  • Hard disk

    • Overhead: AOF and RDB file writing, which can be combined with iostat and iotao analysis
    • Optimization:

      • Do not deploy with high hard disk load service: storage service, message queue;
      • no-appendfsync-on-rewrite=yes;
      • Determine the disk type according to the amount of writes: for example, SDD;
      • Partition can be considered for single machine multi instance persistent file directory;
Aof add blocking

Aof blocking location

  • Redis log
Asynchronous AOF fsync is taking to long(disk is busy?). Writing the AOF 
buffer whitout waiting for fsync to complete, this may slow down Redis
  • info persistence
    You can view the number of times the above logs occurred:
127.0.0.1:6379> info persistence
......
......
aof_delayed_fsync: 100
......
......

Ways of improvement

Hard disk optimization of the same subprocess

PS: more articles please pay attention to WeChat public number: buoyancy.