Stop writes on bgsave error | issues related to redis persistence


I was very sleepy in the afternoon, but when I returned the train and found an error, I was not sleepy.

Well, I’m going to store the key in redis, and then:

(error) MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.

Translated as:

(error) misconf redis has been configured to save RDB snapshots, but it cannot be persisted on the disk at present. Commands that can modify datasets are disabled because if an RDB snapshot fails (stop writes on bgsave error option), this instance is configured to report errors during writes. For details about RDB errors, please check the redis logs.

Just set stop writes on bgsave error to No??? (yes in my configuration file)
What exactly does this field mean?

Let’s start with something else:
Redis can save the database snapshot in dump RDB binary file.
RDB persistence is called snapshot.
operation mode:

  • Redis calls forks to obtain child processes
  • The child process writes the data set to a temporary RDB file
  • When the child process finishes writing the new RDB file, redis replaces the original file with the new RDB file and deletes the old RDB file

This way of working enables redis to benefit from the copy on write mechanism. (AOF rewriting also takes advantage of write time replication)

Copy on writeIt is an optimization strategy in the field of computer programming. The core idea is that if multiple callers request the same resource at the same time, they will jointly obtain the same pointer to the same resource. Until a caller attempts to modify the content of the resource, the system will truly copy a dedicated copy to the caller, while the original resource seen by other callers will remain unchanged. This process is transparent to other callers. The main advantage of this method is that if the caller does not modify the resource, no copy will be created. Therefore, multiple callers can share the same resource only during the read operation.

Actually, stop writes on bgsave error is in the configuration file and in the snapshot section.

################################ SNAPSHOTTING  ################################
# Save the DB on disk:
#   save <seconds> <changes>
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#   In the example below the behavior will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#   Note: you can disable saving completely by commenting out all "save" lines.
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#   save ""

save 900 1
save 300 10
save 60 10000

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
# If the background saving process will start working again Redis will
# automatically allow writes again.
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes

# Compress string objects using LZF when dump .rdb databases?
# By default compression is enabled as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes

# The filename where to dump the DB
dbfilename dump.rdb

# Remove RDB files used by replication in instances without persistence
# enabled. By default this option is disabled, however there are environments
# where for regulations or other security concerns, RDB files persisted on
# disk by masters in order to feed replicas, or stored on disk by replicas
# in order to load them for the initial synchronization, should be deleted
# ASAP. Note that this option ONLY WORKS in instances that have both AOF
# and RDB persistence disabled, otherwise is completely ignored.
# An alternative (and sometimes better) way to obtain the same effect is
# to use diskless replication on both master and replicas instances. However
# in the case of replicas, diskless is not always an option.
rdb-del-sync-files no

# The working directory.
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
# The Append Only File will also be created inside this directory.
# Note that you must specify a directory here, not a file name.
dir ./

By default, if RDB snapshots (at least one savepoint) are enabled and the latest background save fails, redis will stop accepting writes.
This will make the user realize (in a difficult way) that the data is not saved correctly on the disk, otherwise it is likely that no one will notice and some disasters will occur.
If the background saving process starts working again, redis will automatically allow writing again.
However, if you have set up proper monitoring of redis server and persistence, you may want to disable this function so that redis can continue to work as usual, even if there are problems such as disk and permissions.

Therefore, simply setting this option to no may still not solve the problem in essence. It just means that I will back up even if an error occurs.
In fact, stopping work when persistence fails is also a protection mechanism and prompt.

Later, I set stop-writes-on-bgsave-error to yes, but no error was reported when I added key again.

Theoretically, you should look at the log, but I can guess that it should be related to memory, because my server only has 4G of memory, and there are several programs at one time, and the memory space is not enough for fork subprocesses

Even if I have a lot of free RAM, the background saving under Linux will fail due to fork() error!
Short answer: echo 1 > /proc/sys/vm/overcommit_ memory:)

It is now long:

The redis background saving mode depends on the write time replication semantics of fork in the modern operating system: redis fork (create child process) is an exact copy of the parent process. The child process dumps the database to disk and eventually exits. Theoretically, the child process should use as much memory as the parent process as a copy, but in fact, due to the write time replication semantics implemented by most modern operating systems, the parent and child processes will share common memory pages. The page is copied only when the child page or parent page changes. Theoretically, all pages may change when the child process is saved. Linux cannot know in advance how much memory the child process will occupy. Therefore, if overcommit_ If memory is set to zero, fork will fail unless there is enough available RAM to actually copy all the parent memory pages. As a result, if you have a 3 GB redis dataset and only 2 GB of available memory, it will fail.

Set overcommit_ A memory of 1 tells Linux to relax and perform fork in a more optimistic way, which is what redis really wants.

Redis FAQs

If you have different opinions, please leave a message

Recommended Today

JS generate guid method

JS generate guid method Globally unique identification(GUID) is an algorithm generatedBinaryCount Reg128 bitsNumber ofidentifier , GUID is mainly used in networks or systems with multiple nodes and computers. Ideally, any computational geometry computer cluster will not generate two identical guids, and the total number of guids is2^128In theory, it is difficult to make two […]