Summary of redis knowledge points

Redis introduction

Redis is a high-performance key value database. Up to 100000 operations per second + QPS

Redis features

Data persistence is supported. The data in memory can be saved on disk and loaded again when restarting
It supports kV type data and other rich data structure storage
Support data backup, that is, data backup in master slave mode

What data structures does redis support

String: string, integer, or floating point number
List: a list that can store multiple identical strings
Set: a set that stores different elements in an unordered order
Hash: hash table, which stores the mapping between key value pairs and is arranged out of order
Zset: ordered collection, storing key value pairs, and orderly arrangement

Differences between redis and Memcache

Comparison item	Redis	Memcache
data structure	Rich data types	Only simple data types are supported
Data consistency	affair	cas
persistence	Snapshot / AOF	I won’t support it
Network IO	Single threaded IO multiplexing	Multithreaded, non blocking IO multiplexing
Memory management mechanism	On site application memory	Pre allocated memory

Publish and subscribe

Publish / subscribe (Pub / sub) is a message communication mode: the sender (PUB) sends messages and the subscriber (sub) receives messages

Persistence strategy

Snapshot persistence

Write all data at a certain time to the hard disk. useBGSAVECommand. With the increase of memory usage, executing bgsave may cause the system to pause for a long time

Aof persistence

Only files are appended. When the write command is executed, the executed write command is copied to the hard disk. Using the AOF policy requires a large number of writes to the hard disk, and the processing speed of redis will be limited by the performance of the hard disk

Redis transaction

Redis > multi # mark the start of transaction
OK
redis> INCR user_ ID # multiple commands are queued in order
QUEUED
redis> INCR user_id
QUEUED
redis> INCR user_id
QUEUED
redis> PING
QUEUED
Redis > exec # execution
1) (integer) 1
2) (integer) 2
3) (integer) 3
4) PONG

In redis transactions, if a command fails to execute, subsequent commands will still be executed
Use discard to cancel a transaction and discard all commands in the transaction block

How to implement distributed lock

Mode 1

tryLock() {
 SETNX Key 1 Seconds}
release() {
 DELETE Key}

Defect: C₁The execution time is too long and the lock is not released actively, C₂In C₁Obtain the lock after the lock timeout, C₁And C₂Both are executed at the same time, which may cause unknown situations such as data inconsistency. If C₁If the execution is completed first, C will be released₂Another C may be caused at this time₃Get lock

Mode II

tryLock() {
 SETNX Key UnixTimestamp Seconds}
release() {
 EVAL ( //LuaScript if redis.call("get", KEYS[1] == ARGV[1]) then return redis.call("del", KEYS[1]) else return 0 end )}

Defect: in extremely high concurrency scenarios (such as red envelope snatching scenarios), there may be UNIX timestamp duplication. The consistency of physical clock in distributed environment cannot be guaranteed, and there may be a problem of UNIX timestamp duplication

Mode III

tryLock() {
 SET Key UniqId Seconds}
release() {
 EVAL ( //LuaScript if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end )}

implementSET key value NXThe effect is equivalent to executionSETNX key value
At present, it is the best distributed lock scheme, but there are still problems in the cluster. Since the data synchronization of redis cluster is asynchronous, assuming that the master node crashes when the data synchronization is not completed after the master node obtains the lock, the lock can still be obtained at the new master node, so multiple clients obtain the lock at the same time

Redis expiration policy and memory elimination mechanism

Expiration Policies

The expiration policy of redis refers to how redis handles when the cached key in redis expires

Timed Expiration: each key setting the expiration time creates a timer, which is cleared immediately when the expiration time expires. Memory friendly, CPU unfriendly
Lazy Expiration: judge whether the key expires when accessing it. If it expires, it will be cleared. CPU friendly, memory unfriendly
Periodic Expiration: scan a certain number of keys in the expires dictionary every certain time to clear the expired keys. Optimal balance of memory and CPU resources

Memory elimination mechanism

[root]# redis-cli config get maxmemory-policy
1) "maxmemory-policy"
2) "noeviction"

Noeviction: an error will be reported for a new write operation
Allkeys LRU: remove the least recently used key
All keys random: remove some keys randomly
Volatile LRU: among the keys with expiration time set, remove the least recently used key
Volatile random: remove some keys randomly from the keys with expiration time set
Volatile TTL: among the keys with expiration time set, the keys with earlier expiration time are removed first

Why is redis single threaded

Redis is a memory based operation. The CPU is not the bottleneck of redis. The most likely bottleneck of redis is memory or network. Moreover, single thread is easy to implement, avoiding unnecessary context switching and competition conditions, and there is no CPU consumption for multi-threaded switching

How to use CPU multi-core

In a single instance, if the operations are o (n) and O (log (n)) complex, the CPU consumption will not be too high. To maximize CPU utilization, a single machine can deploy multiple instances

Implementation method of set command

command	Implementation of intset coding	Implementation of hashtable coding
SADD	Call the intsetadd function to add all new elements to the integer set	Call dictadd to add the key value pair to the dictionary with the new element as the key and null as the value
SCARD	Call the intsetlen function to return the number of elements contained in the integer collection, which is the number of elements contained in the collection object	Call the dictsize function to return the number of key value pairs contained in the dictionary, which is the number of elements contained in the collection object
SISMEMBER	Call the intsetfind function to find the given element in the integer set. If the element is found in the set, it means that the element does not exist in the set	Call the dictfind function to find the given element in the dictionary key. If it is found, it indicates that the element exists in the collection. If it is not found, it indicates that the element does not exist in the collection
SMEMBERS	Traverse the entire integer set and call the inisetget function to return the set elements	Traverse the whole dictionary, and use the dictgetkey function to return the key of the dictionary as the collection element
SRANDMEMBER	Call the intsetrandom function to return an element randomly from the set of integers	Call the dictgetrandomkey function to randomly return a dictionary key from the dictionary
SPOP	Call the intsetrandom function to randomly take an element from the integer set, and then return the random element to the client. Call the intsetremove function to delete the random element from the integer set	Call the dictgetrandomkey function to randomly take a dictionary key from the dictionary. After returning the value of the random dictionary key to the client, call the dictdelete function to delete the key value pair corresponding to the random dictionary key from the dictionary
SREM	Call the intsetremove function to remove all the given elements from the integer collection	Call the dictdelete function to delete all key value pairs whose keys are given elements from the dictionary

Implementation method of ordered set command

command	Implementation of zip list coding	Implementation of Zset coding
ZADD	Call the ziplistinsert function to insert the member and score into the compressed list as two nodes respectively	First call the zslinsert function to add the new element to the jump table, and then call the dictadd function to associate the new element to the dictionary
ZCARD	Call the ziplistlen function to obtain the number of nodes contained in the compressed list, and divide this number by 2 to obtain the number of collection elements	Access the length attribute of the jump table data structure and directly access the number of collection elements
ZCOUND	Traverse the compressed list and count the number of nodes with scores within a given range	Traverse the jump table and count the number of nodes with scores within a given range
ZRANGE	Traverses the compressed list from the header to the footer, returning all elements within the given index range	Traverses the jump table from the header to the footer, returning all elements within the given index range
ZREVRANGE	The tail of the table traverses the compressed list to the header and returns all elements within the given index range	Traverses the jump table from the end of the table to the header, returning all elements of the given index range
ZRANK	Traverse the compressed list from the header to the footer to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member	Traverse the jump table from the header to the footer to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member
ZREVRANK	Traverse the compressed list from the end of the table to the header to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member	Traverse the jump table from the end of the table to the header, find a given member, and record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member
ZREM	Traverse the compressed list and delete all nodes containing the given member and the score node next to the deleted member node	Traverse the jump table and delete all jump table nodes containing the given members. And disassociate the member and score of the deleted element in the dictionary
ZSCORE	Traverse the compressed list, find the node containing the given member, and then take out the element score saved by the score node next to the member node	Take the score of a given member directly from the dictionary