Redis | Chapter 4 redis database design and Implementation

Time:2021-11-30


preface

reference material: redis design and Implementation Second Edition;

The second part is the implementation of stand-alone database, which is mainly composed of the following modules:databasePersistenceeventclientAndThe server

This chapter will introduce the database in redis;


1. Database in redis

  • All databases of redis server are saved inredis.h/redisServiceIn DB array of structure:

    struct redisService{
        //...
        //Save an array of all databases
        redisDB *db;
      
        //Number of databases on the server
        int dbnum;
        //...
    }
    • dbnumThe attribute is determined by the database option configured by the server. The default value is 16;
  • Redis client database is saved inredisClientStructuraldbProperties:

    typedef struct redisClient{
        //...
        //Records the database currently in use by the client
        redisDB *db;
    } redisClient;
    • The client modifies the target database pointer to point to different elements in the redisservice.db array;
    • Can passSELECT indexCommand to switch the database;
      客户端的目标数据库
  • The database is defined inredis.h/redisDbIn the structure:

    typedef struct redisDb{
        //...
        //Database key space, which stores all key value pairs in the database
        dict *dict;
        
        //The expiration dictionary stores the expiration time of the key
        dict *expires;
    } redisDb;
    • The key of the key space is the key of the database, and each key is a string object;
    • The value of the key space is the value of the database, and each value can beString objectList objectHash table objectCollection objectandOrdered collection objectOne of;

键空间示例

2. Key space of database

  • Database key space is a dictionary. All operations on the database are operated through the key space dictionary;
  • Redis also performs some maintenance operations when reading and writing key spaces:
    • After reading the key, the keyspace will be hit according to whether the key exists or not_ Hits or keyspace misses keyspace_ Number of misses. adoptINFO statsCommand to view properties;
    • After reading the key, the server updates the LRU (last used time) of the key. adoptONBJECT idlettime [key]Command to view the idle time of the key;
    • If the server finds that the key has expired when reading the key, it will delete the expired key first;
    • If a client usesWATCHCommand to monitor a key. After the server modifies the key, it will be marked dirtydirty, draw the attention of the transaction tour;
    • Every time the server modifies a key, it will be dirtydirtyWhen the key counter value increases by 1, the counter will trigger the persistence and replication operations of the server;

3. Key generation time and expiration time

  • Refer to the time related setting command of keySummary of common redis commands and examples 1.3 function of lifetime
  • Lifetime:
    • The client canEXPIREorPEXPIRECommand sets the time to live (TTL) for a key in the database in seconds or milliseconds. After the specified time, the server will automatically delete the key with a lifetime of 0;
    • Can passSETEXCommand sets the expiration time while setting the string key;
    • useTTLorPTTLCommand to obtain the remaining lifetime of the key;
  • Expiration time:
    • The client command to set the expiration time isEXPIREATorPEXPIREAT
    • The expiration time is a UNIX timestamp;
  • EXPIREPEXPIREandEXPIREATAll three commands are converted toPEXPIREATCommand implementation;

4. Expired key deletion policy in redis

  • There are three strategies for deleting expired keys:

    • Scheduled deletion: proactive strategy. It is the most memory friendly and releases memory when it expires. The disadvantage is that it is not friendly to CPU events. When there are many expired keys, it will occupy part of CPU time;
    • Inert deletion: passive strategy. The most CPU friendly. Each time a key is obtained from the key space, check whether the obtained key expires, and delete it if it expires. The disadvantage is that it is not memory friendly;
    • Periodic deletion: proactive strategy. The integration and compromise of the first two strategies. Delete the expired key every other period of time, and reduce the impact of deletion on CPU time by limiting the execution time and frequency of deletion. The disadvantage is that it is difficult to determine the duration and frequency of deletion operation;
  • Inert deletion policydb.c/expireIfNeededThe function implementation process is as follows:

惰性删除策略的执行流程

  • Periodically delete policies byredis.c/activeExpireCycleFunction implementation whenever the redis server operates periodicallyredis.c/serverCronWhen the function is executed, the activeexpirecycle function is called. Within the specified time, traverse each database of the server several times, from the databaseexpiresRandomly check part of the expiration time in the dictionary and delete the expiration key;

5. Processing of expired keys by AOF, RDB and replication functions

5.1 generating RDB files

  • In executionSAVEorBGSAVEWhen the command creates a new RBG file, the program will check the keys in the database, and the expired keys will not be saved to the newly created RDB file;

5.2 loading RDB files

  • When loading RDB files:
    • If the server runs in the main server mode, the expiration key will not be loaded;
    • If the server is running in slave mode, all keys will be loaded;

5.3 AOF file writing

  • When the server runs in AOF persistence mode, the expired keys that are not deleted lazily and periodically will not affect the AOF file;
  • When the expired key is deleted lazily or periodically, the program will append a del command to the AOF file to show that the key has been deleted;

5.4 AOF rewrite

  • Similar to RDB files, during AOF rewriting, the program will check the keys in the database, and the expired keys will not be saved to the rewritten AOF file;

5.5 reproduction

  • When the server is running in the replication mode, the deletion of the expiration key from the server is controlled by the master server;
    客户端访问从服务器
    客户端访问主服务器

6. Database notification

  • Database notification is a new function in redis version 2.8;
  • Database notification allows the client to know the changes of keys in the database and the execution of commands in the database by subscribing to a given channel or mode;
  • After the redis command modifies the database, the server will send a database notification to the client according to the configuration;
  • The relevant commands of the notification can be referred toSummary of common redis commands and examples 7. Pub / sub (publish / subscribe)
  • There are two types of notification:
    • Key space notification: focus on “what command is executed by a key”;SUBSCRIBE channel:message
    • Key event notification: focus on “what key executed a command”;SUBSCRIBE channel:del
  • Server configurednotify-keyspace-eventsOption determines the type of notification sent by the server:
    • Ake: send all types of key space and key event notifications;
    • AK: send all types of key space notifications;
    • AE: send all types of key event notifications;
    • K $: send only the key space notification related to the string;
    • El: send only key event notifications related to list keys;
  • The function of sending database notifications isnotify.c/notifyKeyspaceEventFunction implementation;


last

Newcomer production, if there are mistakes, welcome to point out, thank you very much!Welcome to the official account and share some more everyday things.If you need to reprint, please mark the source!
Redis | Chapter 4 redis database design and Implementation