Redis introduction
Redis is a high-performance key value database. Up to 100000 operations per second + QPS
Redis features
- Data persistence is supported. The data in memory can be saved on disk and loaded again when restarting
- It supports kV type data and other rich data structure storage
- Support data backup, that is, data backup in master slave mode
What data structures does redis support
- String: string, integer, or floating point number
- List: a list that can store multiple identical strings
- Set: a set that stores different elements in an unordered order
- Hash: hash table, which stores the mapping between key value pairs and is arranged out of order
- Zset: ordered collection, storing key value pairs, and orderly arrangement
Differences between redis and Memcache
Comparison item | Redis | Memcache |
---|---|---|
data structure | Rich data types | Only simple data types are supported |
Data consistency | affair | cas |
persistence | Snapshot / AOF | I won’t support it |
Network IO | Single threaded IO multiplexing | Multithreaded, non blocking IO multiplexing |
Memory management mechanism | On site application memory | Pre allocated memory |
Publish and subscribe
Publish / subscribe (Pub / sub) is a message communication mode: the sender (PUB) sends messages and the subscriber (sub) receives messages
Persistence strategy
Snapshot persistence
Write all data at a certain time to the hard disk. useBGSAVE
Command. With the increase of memory usage, executing bgsave may cause the system to pause for a long time
Aof persistence
Only files are appended. When the write command is executed, the executed write command is copied to the hard disk. Using the AOF policy requires a large number of writes to the hard disk, and the processing speed of redis will be limited by the performance of the hard disk
Redis transaction
Redis > multi # mark the start of transaction
OK
redis> INCR user_ ID # multiple commands are queued in order
QUEUED
redis> INCR user_id
QUEUED
redis> INCR user_id
QUEUED
redis> PING
QUEUED
Redis > exec # execution
1) (integer) 1
2) (integer) 2
3) (integer) 3
4) PONG
In redis transactions, if a command fails to execute, subsequent commands will still be executed
Use discard to cancel a transaction and discard all commands in the transaction block
How to implement distributed lock
Mode 1
tryLock() {
SETNX Key 1 Seconds}
release() {
DELETE Key}
Defect: C1The execution time is too long and the lock is not released actively, C2In C1Obtain the lock after the lock timeout, C1And C2Both are executed at the same time, which may cause unknown situations such as data inconsistency. If C1If the execution is completed first, C will be released2Another C may be caused at this time3Get lock
Mode II
tryLock() {
SETNX Key UnixTimestamp Seconds}
release() {
EVAL ( //LuaScript if redis.call("get", KEYS[1] == ARGV[1]) then return redis.call("del", KEYS[1]) else return 0 end )}
Defect: in extremely high concurrency scenarios (such as red envelope snatching scenarios), there may be UNIX timestamp duplication. The consistency of physical clock in distributed environment cannot be guaranteed, and there may be a problem of UNIX timestamp duplication
Mode III
tryLock() {
SET Key UniqId Seconds}
release() {
EVAL ( //LuaScript if redis.call("get", KEYS[1]) == ARGV[1] then return redis.call("del", KEYS[1]) else return 0 end )}
implement
SET key value NX
The effect is equivalent to executionSETNX key value
At present, it is the best distributed lock scheme, but there are still problems in the cluster. Since the data synchronization of redis cluster is asynchronous, assuming that the master node crashes when the data synchronization is not completed after the master node obtains the lock, the lock can still be obtained at the new master node, so multiple clients obtain the lock at the same time
Redis expiration policy and memory elimination mechanism
Expiration Policies
The expiration policy of redis refers to how redis handles when the cached key in redis expires
- Timed Expiration: each key setting the expiration time creates a timer, which is cleared immediately when the expiration time expires. Memory friendly, CPU unfriendly
- Lazy Expiration: judge whether the key expires when accessing it. If it expires, it will be cleared. CPU friendly, memory unfriendly
- Periodic Expiration: scan a certain number of keys in the expires dictionary every certain time to clear the expired keys. Optimal balance of memory and CPU resources
Memory elimination mechanism
[root]# redis-cli config get maxmemory-policy
1) "maxmemory-policy"
2) "noeviction"
- Noeviction: an error will be reported for a new write operation
- Allkeys LRU: remove the least recently used key
- All keys random: remove some keys randomly
- Volatile LRU: among the keys with expiration time set, remove the least recently used key
- Volatile random: remove some keys randomly from the keys with expiration time set
- Volatile TTL: among the keys with expiration time set, the keys with earlier expiration time are removed first
Why is redis single threaded
Redis is a memory based operation. The CPU is not the bottleneck of redis. The most likely bottleneck of redis is memory or network. Moreover, single thread is easy to implement, avoiding unnecessary context switching and competition conditions, and there is no CPU consumption for multi-threaded switching
How to use CPU multi-core
In a single instance, if the operations are o (n) and O (log (n)) complex, the CPU consumption will not be too high. To maximize CPU utilization, a single machine can deploy multiple instances
Implementation method of set command
command | Implementation of intset coding | Implementation of hashtable coding |
---|---|---|
SADD | Call the intsetadd function to add all new elements to the integer set | Call dictadd to add the key value pair to the dictionary with the new element as the key and null as the value |
SCARD | Call the intsetlen function to return the number of elements contained in the integer collection, which is the number of elements contained in the collection object | Call the dictsize function to return the number of key value pairs contained in the dictionary, which is the number of elements contained in the collection object |
SISMEMBER | Call the intsetfind function to find the given element in the integer set. If the element is found in the set, it means that the element does not exist in the set | Call the dictfind function to find the given element in the dictionary key. If it is found, it indicates that the element exists in the collection. If it is not found, it indicates that the element does not exist in the collection |
SMEMBERS | Traverse the entire integer set and call the inisetget function to return the set elements | Traverse the whole dictionary, and use the dictgetkey function to return the key of the dictionary as the collection element |
SRANDMEMBER | Call the intsetrandom function to return an element randomly from the set of integers | Call the dictgetrandomkey function to randomly return a dictionary key from the dictionary |
SPOP | Call the intsetrandom function to randomly take an element from the integer set, and then return the random element to the client. Call the intsetremove function to delete the random element from the integer set | Call the dictgetrandomkey function to randomly take a dictionary key from the dictionary. After returning the value of the random dictionary key to the client, call the dictdelete function to delete the key value pair corresponding to the random dictionary key from the dictionary |
SREM | Call the intsetremove function to remove all the given elements from the integer collection | Call the dictdelete function to delete all key value pairs whose keys are given elements from the dictionary |
Implementation method of ordered set command
command | Implementation of zip list coding | Implementation of Zset coding |
---|---|---|
ZADD | Call the ziplistinsert function to insert the member and score into the compressed list as two nodes respectively | First call the zslinsert function to add the new element to the jump table, and then call the dictadd function to associate the new element to the dictionary |
ZCARD | Call the ziplistlen function to obtain the number of nodes contained in the compressed list, and divide this number by 2 to obtain the number of collection elements | Access the length attribute of the jump table data structure and directly access the number of collection elements |
ZCOUND | Traverse the compressed list and count the number of nodes with scores within a given range | Traverse the jump table and count the number of nodes with scores within a given range |
ZRANGE | Traverses the compressed list from the header to the footer, returning all elements within the given index range | Traverses the jump table from the header to the footer, returning all elements within the given index range |
ZREVRANGE | The tail of the table traverses the compressed list to the header and returns all elements within the given index range | Traverses the jump table from the end of the table to the header, returning all elements of the given index range |
ZRANK | Traverse the compressed list from the header to the footer to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member | Traverse the jump table from the header to the footer to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member |
ZREVRANK | Traverse the compressed list from the end of the table to the header to find a given member. Record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member | Traverse the jump table from the end of the table to the header, find a given member, and record the number of nodes along the way. When a given member is found, the number of nodes along the way is the ranking of the elements corresponding to the member |
ZREM | Traverse the compressed list and delete all nodes containing the given member and the score node next to the deleted member node | Traverse the jump table and delete all jump table nodes containing the given members. And disassociate the member and score of the deleted element in the dictionary |
ZSCORE | Traverse the compressed list, find the node containing the given member, and then take out the element score saved by the score node next to the member node | Take the score of a given member directly from the dictionary |