[Redis5 source code learning] analysis of the randomkey part of redis command

Time:2020-4-5

baiyan

Command syntax

Command meaning: randomly return a key from the currently selected database
Command format:

RANDOMKEY

Command actual combat:

127.0.0.1:6379> keys *
1) "kkk"
2) "key1"

127.0.0.1:6379> randomkey
"key1"
127.0.0.1:6379> randomkey
"kkk"

Return value: random key; nil if database is empty

Source code analysis

Main process

The processing function corresponding to the keys command is randomkeycommand():

void randomkeyCommand(client *c) {
    Robj * key; // store the obtained key

    If ((key = dbrandomkey (C - > dB)) = = null) {// call the core function dbrandomkey ()
        Addreply (C, shared. Nullbulk); // returns nil
        return;
    }

    Addreplybulk (C, key); // return key
    Decrefcount (key); // reduce the reference count
}

Random key generation and expiration judgment

Randomkeycommand() calls the dbrandomkey() function to actually generate a random key:

robj *dbRandomKey(redisDb *db) {
    dictEntry *de;
    int maxtries = 100;
    int allvolatile = dictSize(db->dict) == dictSize(db->expires);

    while(1) {
        sds key;
        robj *keyobj;

        De = dictgetrandomkey (DB - > dict); // get a random dictentry
        If (de = = null) return null; // get failure returns null

        Key = dictgetkey (DE); // get the key in dictentry
        Keyobj = createstringobject (key, sdslen (key)); // generate robj based on key string
        If (dictfind (DB - > expires, key)) {// find the key in the expiration dictionary
            ...
            If (expireifneed (DB, keyobj)) {// judge whether the key is expired
                Decrefcount (keyobj); // if expired, delete the key and reduce the reference count
                Continue; // the current key is expired and cannot be returned. Only the keys that are not expired will be returned for the next random generation
            }
        }
        return keyobj;
    }
}

Then the main logic of this layer calls dictgetrandomkey() to get a random dictentry. Suppose we have obtained the randomly generated dictentry, then we take out the key. Because expired keys cannot be returned, we need to first determine whether the keys are expired. If they are expired, they cannot be returned. Continue directly. If they are not expired, they can be returned.

An algorithm for real random key acquisition

Then we will continue to follow up the dictgetrandomkey() function to see what algorithm is used to generate dictentry randomly:

dictEntry *dictGetRandomKey(dict *d)
{
    dictEntry *he, *orighe;
    unsigned long h;
    int listlen, listele;

    If (dictsize (d) = = 0) return null; // the dictionary passed in is empty and does not need to be generated at all
    If (dictisrehashing (d))] dictrehashstep (d); // perform a rehash operation
    If (dictisrehashing (d)) {// if rehash is in progress, be sure to evenly distribute random seeds from two hash tables
        do {
            H = D - > rehashidx + (random()% (D - > HT [0]. Size + D - > HT [1]. Size - D - > rehashidx)); // calculate the random hash value, which must be at the back of rehashidx
            He = (H > = D - > HT [0]. Size)? D - > HT [1]. Table [H - D - > HT [0]. Size]: D - > HT [0]. Table [H]; // get the corresponding bucket according to the hash value calculated above
        }While (he = = null); // the last bucket whose calculation result is not empty is taken as the cycle calculation
    }Else {// not in rehash, only one hash table
        do {
            H = random() & D - > HT [0]. Sizemask; // directly calculate the hash value
            He = D - > HT [0]. Table [H]; // retrieve the h bucket on the hash table
        }While (he = = null); // the last bucket whose calculation result is not empty is taken as the cycle calculation
    }

    //Now we get a bucket that is not empty, and one or more dictentries are attached to the back of the bucket (the chain address method solves the hash conflict), so we also need to calculate a random index to determine which dickentry chain node to access
    listlen = 0;
    orighe = he;
    while(he) {
        he = he->next;
        Listlen + +; // calculate the length of the linked list
    }
    Listele = random()% listlen; // the random number takes the remainder of the length of the linked list and determines which node to get
    he = orighe;
    While (listele --) he = He - > next; // traverse the linked list on the bucket from the front to the back and find the node
    Return he; // finally return this node
}

This function first determines that the dictionary is empty. Then a single step rehash operation will be carried out, which is the same as the effect of calling dictionary functions such as dictadd(), and is part of the progressive rehash technology. Here we first review the overall structure of the dictionary:
[Redis5 source code learning] analysis of the randomkey part of redis command
Because rehash will affect the generation of random number seed, there are two situations to discuss according to whether the current dictionary is in rehash operation:
First: rehash operation in progress.Then the structure of the current dictionary is: there are some keys on the first hash table and the rest on the second hash table. In order to evenly allocate the probability that two hash tables may be fetched, it is necessary to combine the two hash tables. The algorithm is as follows:

H = D - > rehashidx + (random()% (D - > HT [0]. Size + D - > HT [1]. Size - D - > rehashidx)); // calculate the random hash value, which must be at the back of rehashidx

Here we subtract rehashidx from the sum of two hash table sizes by a random number. Such a residual operation can ensure that the hash value will randomly fall into the indexgreater thanOn the bucket at rehashidx. Because rehashidx represents the progress of rehash. This rehashidx represents the data before the index in the first hash table, i.e. [0, rehashidx-1]. The data in this closed interval has been rehashed to the second hash table. The elements greater than or equal to this rehashidx are still on the first hash table. Therefore, this ensures that any bucket on result h is non empty and has a value. Next, you only need to determine which hash table this H value is in, and then go to the bucket value at the corresponding position in the hash table:

he = (h >= d->ht[0].size) ? d->ht[1].table[h - d->ht[0].size] : d->ht[0].table[h];

Second: no rehash operation.Then all keys are on the only first dictionary, which is very simple. You can directly calculate the length of the dictionary, or perform bitwise and operation on the sizemask of the dictionary, which can ensure that the calculated results fall in the hash table. Redis chooses the latter:

H = random() & D - > HT [0]. Sizemask; // hash value is calculated by bitwise and operation of sizemask
He = D - > HT [0]. Table [H]; // retrieve the h bucket on the hash table

Next, we find a non empty bucket, but it’s not over yet. Because there may beHash CollisionsRedis uses the chain address method to solve hash conflicts, so multiple dictentries will be attached to a bucket to form a chain list. Therefore, we also need to think about which dictentry on the linked list node to take. This algorithm is relatively simple. You can directly use the result of random() to calculate the length of the linked list

Listele = random()% listlen; // the random number takes the remainder of the length of the linked list and determines which node to get
While (listele --) he = He - > next; // traverse the linked list on the bucket from the front to the back and find the node

So far, we have found a random dictentry node on a random bucket, so we can return it to the client.

Recommended Today

Configure Apache to support PHP in the Apache main configuration file httpd.conf Include custom profile in

In Apache’s main configuration file / conf/ http.conf Add at the bottom Include “D:workspace_phpapache-php.conf” The file path can be any In D: workspace_ Create under PHP file apache- php.conf file Its specific content is [html] view plain copy PHP-Module setup LoadFile “D:/xampp/php/php5ts.dll” LoadModule php5_module “D:/xampp/php/php5apache2_2.dll” <FilesMatch “.php$”> SetHandler application/x-httpd-php </FilesMatch> <FilesMatch “.phps$”> SetHandler application/x-httpd-php-source </FilesMatch> […]