Redis actual combat — 14. Lua script programming of redis

Time:2021-2-7

brief introduction

Redis has introduced the server-side script programming function using Lua programming language since version 2.6. This function allows users to perform various operations directly inside redis, so as to simplify the code and improve the performance.P248

Adding new features without writing C codeP248

By using Lua to script redis, we can avoid some common pitfalls that slow down the development speed or cause performance degradation.P248

Load Lua script into redisP249
  • SCRIPT LOADThe command can load the script into redis. This command takes a Lua script in string format as a parameter, stores the script for later use, and then returns the SHA1 check sum of the stored script
  • EVALSHAThe command can call the previously stored script. This command receives the SHA1 checksums of the script and all the parameters required by the script
  • EVALCommand can directly execute the specified script. This command receives the script string and all the parameters required by the script. This command will not only execute the script, but also cache the executed script to the redis server

Due to the limitation of lua’s data in and out, Lua and redis need to transform each other. Because scripts may produce ambiguous results when returning various types of data, we should try to return strings explicitly.P250

We can find the conversion table between different types of redis and Lua in the official documents

Convert redis type to Lua type

Redis Lua
integer reply number
bulk reply string
multi bulk reply table (may have other Redis data types nested)
status reply table with a single ok field containing the status
error reply table with a single err field containing the error
Nil bulk reply false boolean type
Nil multi bulk reply false boolean type

Lua type to redis type

Lua Redis
number integer reply (the number is converted into an integer)
string bulk reply
table (array) multi bulk reply (truncated to the first nil inside the Lua array if any)
table with a single ok field status reply
table with a single err field error reply
boolean false Nil bulk reply
boolean true integer reply with value of 1
Create a new status messageP251
  • Lua script and single redis command as well asMULTI/EXECLike transactions, they are all atomic operations
  • Lua scripts that have changed the structure cannot be interrupted

    • Do not execute any write command on read-only script: the script pair can be run more thanlua-time-limitAfter the time specified by theSCRIPT KILLCommand is running on the script
    • Script with write command: killing the script will cause the data stored in redis to enter an inconsistent state. under these circumstances

Using Lua to rewrite locks and semaphoresP254

If we don’t know which keys will be read and written in advance, we should useWATCH/MULTI/EXECTransactions or locks, not Lua scripts. Therefore, in the script, it is not recordedKEYSIf the key in the parameter is read or written, incompatibility or failure may occur when the program is migrated to redis cluster.P254

At present, Lua script is not needed to obtain lock, so it can be used directlySET, andPXandNXOption to set the value with expiration time when the key does not exist. In order to release the lock, Lua script is used to realize it. The related codes have been implemented in the implementation of automatic completion, distributed lock and counting semaphore.

removeWATCH/MULTI/EXECaffairP258

In general, if only a few clients try toWATCHCommand monitoring modifies data, so transactions can usually be completed without obvious conflicts or retries. However, if the operation requires several round trips, or the probability of operation conflict is high, or the network delay is large, the client may need to try again many times to complete the operation.P258

Using Lua script instead of transaction can not only ensure the successful execution of client attempts, but also reduce communication overhead and greatly improve TPS. At the same time, because Lua script does not have multiple round trips, the execution speed will be significantly faster than the fine-grained lock version.

Lua script can provide huge performance advantages, and can greatly simplify the code in some cases, but it runs inside redis, but Lua script can only access the data in Lua script or redis database, and lock or delete the dataWATCH/MULTI/EXECTransactions do not have this limitation.P263

Use Lua to slice the listP263

Composition of partition listP263

In order to perform push and pop operations on both ends of the partition list, in addition to storing each partition that constitutes the list, the ID of the first partition and the ID of the last partition in the list need to be recorded when building the partition list. When the fragment list is empty, the fragment IDs stored in these two strings will be the same.P263

Each fragment that makes up the fragment list is named<listname>:<shardid>And distribute them in order. Specifically, if the program always ejects elements from the left and pushes elements from the right, the index of the last allocation will gradually increase, and the ID of the new partition will become larger and larger. If the program always ejects elements from the right and pushes elements from the left, the index of the first partition will gradually decrease, and the ID of the new partition will become smaller and smaller.P264

When a partitioned list contains multiple lists, the lists at both ends of the partitioned list may be filled, but other lists between the two ends are always filled.P264

Push elements into the sliced listP265

Lua script based on commandLPUSH/RPUSHFind the first partition or the last partition of the list, and then push the element into the list corresponding to the partition. If the partition has reached the maximum number (you can take the value in the configuration)list-max-ziplist-entriesThen, a new partition will be automatically generated, continue to push, and update the partition ID of the first or last partition. When the push operation is finished, it returns the number of pushed elements.P265

Eject elements from the tilesP266

Lua script based on commandLPOP/RPOPFind the first partition or the last partition of the list, and then pop up an element from the partition when the partition is not empty. If the list no longer contains any elements after the pop-up operation, the program will modify the character string key recording the partition information at both ends of the list (note that only when the partition at the end of the list is empty, the corresponding string key will be modified, and the When the whole list is empty, no adjustment will be made.)P267

Perform block pop-up operation on fragment listP267

I can’t understand this passage in the book, and I don’t know why I need the fancy operation in the book to complete it.

In my opinion, blocking pop-up of partitioned list does not need blocking pop-up of the list itself. We can continue to perform the pop-up element operation implemented by Lua script. If the pop-up is successful, we will return directly. If the pop-up fails, we will continue to perform the pop-up operation after sleeping for 1 ms until the pop-up is successful or the timeout is reached. In this way, we can only operate redis in Lua scripts. Atomicity ensures that elements at both ends of the fragment list will pop up.

This article starts with the official account: full Fu machine (click to view the original), open source in GitHub:reading-notes/redis-in-action
Redis actual combat -- 14. Lua script programming of redis