Summary of redis knowledge points

Time:2021-9-23

Redis introduction

Redis is a high-performance key value database. Up to 100000 operations per second + QPS

Redis features

  • Data persistence is supported. The data in memory can be saved on disk and loaded again when restarting
  • It supports kV type data and other rich data structure storage
  • Support data backup, that is, data backup in master slave mode

What data structures does redis support

  • String: string, integer, or floating point number
  • List: a list that can store multiple identical strings
  • Set: a set that stores different elements in an unordered order
  • Hash: hash table, which stores the mapping between key value pairs and is arranged out of order
  • Zset: ordered collection, storing key value pairs, and orderly arrangement

Differences between redis and Memcache

Comparison item Redis Memcache
data structure Rich data types Only simple data types are supported
Data consistency affair cas
persistence Snapshot / AOF I won’t support it
Network IO Single threaded IO multiplexing Multithreaded, non blocking IO multiplexing
Memory management mechanism On site application memory Pre allocated memory

Publish and subscribe

Publish / subscribe (Pub / sub) is a message communication mode: the sender (PUB) sends messages and the subscriber (sub) receives messages

Persistence strategy

Snapshot persistence

Write all data at a certain time to the hard disk. useBGSAVECommand. With the increase of memory usage, executing bgsave may cause the system to pause for a long time

Aof persistence

Only files are appended. When the write command is executed, the executed write command is copied to the hard disk. Using the AOF policy requires a large number of writes to the hard disk, and the processing speed of redis will be limited by the performance of the hard disk

Redis transaction

Redis > multi # mark the start of transaction
OK
redis> INCR user_ ID # multiple commands are queued in order
QUEUED
redis> INCR user_id
QUEUED
redis> INCR user_id
QUEUED
redis> PING
QUEUED
Redis > exec # execution
1) (integer) 1
2) (integer) 2
3) (integer) 3
4) PONG

In redis transactions, if a command fails to execute, subsequent commands will still be executed
Use discard to cancel a transaction and discard all commands in the transaction block

How to implement distributed lock

Mode 1

tryLock() {
 SETNX Key 1 Seconds}
release() {
 DELETE Key}

Defect: C1The execution time is too long and the lock is not released actively, C2In C1Obtain the lock after the lock timeout, C1And C2Both are executed at the same time, which may cause unknown situations such as data inconsistency. If C1If the execution is completed first, C will be released2Another C may be caused at this time3Get lock

Mode II

tryLock() {
 SETNX Key UnixTimestamp Seconds}
release() {
 EVAL ( //LuaScript if redis.call("get", KEYS[1] == ARGV[1]) then return redis.call("del", KEYS[1]) else return 0 end )}

Defect: in extremely high concurrency scenarios (such as red envelope snatching scenarios), there may be UNIX timestamp duplication. The consistency of physical clock in distributed environment cannot be guaranteed, and there may be a problem of UNIX timestamp duplication

Mode III

tryLock() {
 SET Key UniqId Seconds}
release() {
 EVAL ( //LuaScript if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end )}

implementSET key value NXThe effect is equivalent to executionSETNX key value
At present, it is the best distributed lock scheme, but there are still problems in the cluster. Since the data synchronization of redis cluster is asynchronous, assuming that the master node crashes when the data synchronization is not completed after the master node obtains the lock, the lock can still be obtained at the new master node, so multiple clients obtain the lock at the same time

Redis expiration policy and memory elimination mechanism

Expiration Policies

The expiration policy of redis refers to how redis handles when the cached key in redis expires

  • Timed Expiration: each key setting the expiration time creates a timer, which is cleared immediately when the expiration time expires. Memory friendly, CPU unfriendly
  • Lazy Expiration: judge whether the key expires when accessing it. If it expires, it will be cleared. CPU friendly, memory unfriendly
  • Periodic Expiration: scan a certain number of keys in the expires dictionary every certain time to clear the expired keys. Optimal balance of memory and CPU resources

Memory elimination mechanism

[root]# redis-cli config get maxmemory-policy
1) "maxmemory-policy"
2) "noeviction"
  • Noeviction: an error will be reported for a new write operation
  • Allkeys LRU: remove the least recently used key
  • All keys random: remove some keys randomly
  • Volatile LRU: among the keys with expiration time set, remove the least recently used key
  • Volatile random: remove some keys randomly from the keys with expiration time set
  • Volatile TTL: among the keys with expiration time set, the keys with earlier expiration time are removed first

Why is redis single threaded

Redis is a memory based operation. The CPU is not the bottleneck of redis. The most likely bottleneck of redis is memory or network. Moreover, single thread is easy to implement, avoiding unnecessary context switching and competition conditions, and there is no CPU consumption for multi-threaded switching

How to use CPU multi-core

In a single instance, if the operations are o (n) and O (log (n)) complex, the CPU consumption will not be too high. To maximize CPU utilization, a single machine can deploy multiple instances

Implementation method of set command

command Implementation of intset coding Implementation of hashtable coding
SADD Call the intsetadd function to add all new elements to the integer set Call dictadd to add the key value pair to the dictionary with the new element as the key and null as the value
SCARD Call the intsetlen function to return the number of elements contained in the integer collection, which is the number of elements contained in the collection object Call the dictsize function to return the number of key value pairs contained in the dictionary, which is the number of elements contained in the collection object
SISMEMBER Call the intsetfind function to find the given element in the integer set. If the element is found in the set, it means that the element does not exist in the set Call the dictfind function to find the given element in the dictionary key. If it is found, it indicates that the element exists in the collection. If it is not found, it indicates that the element does not exist in the collection
SMEMBERS Traverse the entire integer set and call the inisetget function to return the set elements Traverse the whole dictionary, and use the dictgetkey function to return the key of the dictionary as the collection element
SRANDMEMBER Call the intsetrandom function to return an element randomly from the set of integers Call the dictgetrandomkey function to randomly return a dictionary key from the dictionary
SPOP Call the intsetrandom function to randomly take an element from the integer set, and then return the random element to the client. Call the intsetremove function to delete the random element from the integer set Call the dictgetrandomkey function to randomly take a dictionary key from the dictionary. After returning the value of the random dictionary key to the client, call the dictdelete function to delete the key value pair corresponding to the random dictionary key from the dictionary
SREM Call the intsetremove function to remove all the given elements from the integer collection Call the dictdelete function to delete all key value pairs whose keys are given elements from the dictionary

Implementation method of ordered set command

command Implementation of zip list coding Implementation of Zset coding
ZADD Call the ziplistinsert function to insert the member and score into the compressed list as two nodes respectively First call the zslinsert function to add the new element to the jump table, and then call the dictadd function to associate the new element to the dictionary
ZCARD Call the ziplistlen function to obtain the number of nodes contained in the compressed list, and divide this number by 2 to obtain the number of collection elements Access the length attribute of the jump table data structure and directly access the number of collection elements
ZCOUND Traverse the compressed list and count the number of nodes with scores within a given range Traverse the jump table and count the number of nodes with scores within a given range
ZRANGE Traverses the compressed list from the header to the footer, returning all elements within the given index range Traverses the jump table from the header to the footer, returning all elements within the given index range
ZREVRANGE The tail of the table traverses the compressed list to the header and returns all elements within the given index range Traverses the jump table from the end of the table to the header, returning all elements of the given index range
ZRANK Traverse the compressed list from the header to the footer to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member Traverse the jump table from the header to the footer to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member
ZREVRANK Traverse the compressed list from the end of the table to the header to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member Traverse the jump table from the end of the table to the header, find a given member, and record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member
ZREM Traverse the compressed list and delete all nodes containing the given member and the score node next to the deleted member node Traverse the jump table and delete all jump table nodes containing the given members. And disassociate the member and score of the deleted element in the dictionary
ZSCORE Traverse the compressed list, find the node containing the given member, and then take out the element score saved by the score node next to the member node Take the score of a given member directly from the dictionary

Recommended Today

Supervisor

Supervisor [note] Supervisor – H view supervisor command help Supervisorctl – H view supervisorctl command help Supervisorctl help view the action command of supervisorctl Supervisorctl help any action to view the use of this action 1. Introduction Supervisor is a process control system. Generally speaking, it can monitor your process. If the process exits abnormally, […]