Aof writing and AOF rewriting block redis and Solutions

Time:2021-7-15

Aof write strategy

Controlled by appendfsync parameter:

Configurable values explain
always After the command is written to buf, the system calls the fsync synchronization AOF file. After fsync completes, the thread returns.
no After the command is written to buf, the system calls write operation. The subsequent fsync synchronization operation is completed by the operating system, usually 30 seconds.
everysec After the command is written to buf, the system calls the write operation. The subsequent fsync synchronizes the operation of the specialized thread once every second.

Everysec is a trade-off between always and No. It’s a combination of performance and security. It’s the default configuration of redis, and it’s also a recommended configuration.

When using everysec configuration, redis will useFsync synchronization is completed by the background sub thread.

Aof rewriting

Aof rewriting is done by the backstage sub thread, but AOF will have a large number of IO operations, and eventually compete with the fsync of AOF for disk.

block

Although the fsync of AOF is operated by the sub thread under everysec configuration, the main thread will monitor the execution progress of fsync.

If the main thread finds that the last fsync operation has not returned, the main thread will block.

Solving the blockage

  1. Reduce IO competition

    Redis has a configuration:

    no-appendfsync-on-rewrite yes/no
    • Configuration is to stop the fsync operation of AOF during AOF rewriting when it is set to yes(No more competition)。 This configuration has a potential risk: if redis goes down during AOF rewriting, AOF data will be lost,Reliability degradation
    • When the configuration is set to no, fsync will still be executed during AOF rewriting. At this time, IO contention will occur, which may block the main thread.
  2. Using SSD

    If both performance and reliability are needed.

    • Set the configuration to No
    • The hard disk adopts high speed SSD

Recommended Today

Implementation example of go operation etcd

etcdIt is an open-source, distributed key value pair data storage system, which provides shared configuration, service registration and discovery. This paper mainly introduces the installation and use of etcd. Etcdetcd introduction etcdIt is an open source and highly available distributed key value storage system developed with go language, which can be used to configure sharing […]