. Net client implements pipeline and transactions in redis

Time:2021-12-27

preface

Pipeline feature in redis: briefly describe how redis sends multiple commands from the client at one time, and how the server responds to multiple commands from the client at one time.

Redis uses the TCP server of the client server model and request / response protocol, which means that a request can be completed only after the following steps: 1. The client sends a query command to the server, and then usually waits for the server to respond in a blocking manner. 2. The server processes the query command and sends it back to the client. In this way, it will be connected through the network. If it is a local loopback interface, it can respond very quickly. However, it will be particularly painful if you go to the external network or even do a series of layer by layer forwarding on the external network. No matter what the network delay is, it will take up the overall response time. In this way, if we send one command at a time and the network delay is 100ms, we have to do it. Then, if 1000 commands are issued at a time, the network delay of 100 * 1000ms is difficult to tolerate.

In view of the above problems, Redis has provided the pipeline function since version 2.6. It enables the client to process new requests without reading the old response. In this way, it can send multiple commands to the server without waiting for a reply until the reply is read in the last step. This is called pipeline (pipeline) and has been widely used for decades. For example, many POP3 protocol implementations have supported this function, which greatly speeds up the process of downloading new e-mail from the server.

So the word “affairs” is often encountered. There is not much chirp. The goal should be the same, that is, how to make a group of operations into atomic operations, so that he can’t go to the end and return to the origin.

Brief introduction of Wireshark packet capture tool

In order to give you a more vivid view of the pipeline, in this section, let’s talk about Wireshark packet capture tool, which will let you see the process and details of redis commands sent from the client to the server through TCP protocol.

Wireshark can capture every message sent and received by the system. We only give a brief description of filtering here. The following figure shows his appearance. You can explore his usage after opening it.

Briefly describe several filtering rules:

1. IP filtering: target IP filtering: IP dst==172.18. 8.11. Source IP address filtering: src==192.168. 1.12;

2. Port filtering: TCP Port = = 80. This rule filters out both the source port and the destination port of 80. Using TCP Dstport = = 80 only filters TCP with the destination port of 80 Srcport = = 80 filter only packets with source port 80;

3. Protocol filtering: directly enter the protocol name in the filter box, such as HTTP, TCP, UDP

4. HTTP mode filtering: filter the get package, http request. Method = = “get”, filter the post package, http request. method==”POST”;

5. If you use multi condition filtering, you need to add a connection symbol, and. For example, IP src==192.168. 1.12 and http. request. method==”POST” and tcp. srcport==80

StackExchange. Redis implements redis pipeline

The last two pictures are clear at a glance.

If the client makes multiple requests to the redis server, the general mode is like this

If the client makes multiple requests to the redis server, the pipeline mode is like this

General mode code:

public static void GetNoPipelining()
  {
   for (var i = 0; i < 3; i++)
   {
    var key = "name:" + i;
    db. Stringappend (key, "Zhang Longhao");
   }
  }

View the data of TCP request message

When you do this yourself, you can see that the keys of the three TCP requests I circle are Name: 0, name: 1 and name: 2.

So we use the pipeline model

public static void GetPipelining()
  {
   var batch = db.CreateBatch();
   for (int i = 0; i < 3; i++)
   {
    var key = "mename:" + i;
    batch. Stringappendasync (key, "Zhang Longhao");
   }
   batch.Execute();
  }

Let’s look at the request again

In this way, it is obvious that one request sends multiple commands. Then we can achieve this effect without createbatch ().


var a = db.StringAppendAsync("zlh:1", "zhanglonghao1");
   var b = db.StringAppendAsync("zlh:2", "zhanglonghao2");
   var c = db.StringAppendAsync("zlh:3", "zhanglonghao3");
   var aa = db.Wait(a);
   var bb = db.Wait(a);
   var cc = db.Wait(a);

Next, we make a simple performance comparison. The code is as follows:

static void Main(string[] args)
  {
   Stopwatch watch = new Stopwatch();
   Stopwatch watch1 = new Stopwatch();
   watch.Start();
   GetNoPipelining();
   Console. Writeline ("general cycle time:" + watch. Elapsedmilliseconds);
   watch.Stop();
   watch1.Start();
   GetPipelining();
   Console. Writeline ("pipelining insertion time:" + watch1. Elapsedmilliseconds);
   watch1.Stop();
   Console.ReadLine();
  }
  public static void GetNoPipelining()
  {
   for (var i = 0; i < 5000; i++)
   {
    var key = "name:" + i;
    db. Stringappend (key, "Zhang Longhao");
   }
  }
  public static void GetPipelining()
  {
   var batch = db.CreateBatch();
   for (int i = 0; i < 5000; i++)
   {
    var key = "mename:" + i;
    batch. Stringappendasync (key, "Zhang Longhao");
   }
   batch.Execute();
  }

The results are as follows:

I’ll also talk about stackeexchange There are three command modes of redis. Sending commands in 2 and 3 modes will be encapsulated in the pipeline by default. If you don’t believe it, you can make a small demo test:

1. Sync: synchronous mode, which blocks the caller directly, but does not block other threads.

2. Async: asynchronous mode, encapsulated by task model.

3. Fire and forget: Send a command, and then don’t care when the command operation is finally completed. In the fire and forget mode, all commands will immediately get the return value, which is the default value of the return value type. For example, if the operation return type is bool, it will immediately get false, because false = default (bool).

Refer to the official redis documentation and stackexchange.com for this section The official redis document is connected as follows:

https://redis.io/topics/pipelining

https://github.com/StackExchange/StackExchange.Redis/blob/master/Docs/PipelinesMultiplexers.md

StackExchange. Redis implements redis transactions

From the official documents, I can only say that the implementation is very strange. Let me first describe my environment, which is to prepare an empty redis library, and then go down step by step. Let’s write code to see the results and do this business.


static void Main(string[] args)
  {
   var tran = db.CreateTransaction();
   tran.AddCondition(Condition.ListIndexNotEqual("zlh:1",0,"zhanglonghao"));  
   tran.ListRightPushAsync("zlh:1", "zhanglonghao"); 
   bool committed = tran.Execute();
   Console.WriteLine(committed);
   Console.ReadLine();
  }

The execution result is: true. The results in the database are as follows, indicating that we inserted successfully.

That is, if the key is: zlh: 1, the value of the list set at the beginning of index 0= For zhanglonghao, we insert a piece of data from the right side of the linked list. The key is zlh: 1 and the value is zhanglonghao. Success. Because the first operation is an empty library. 0 is really not Zhang Longhao.

If the data is not cleared, continue with the code.


static void Main(string[] args)
  {
   var tran = db.CreateTransaction();
   tran.AddCondition(Condition.ListIndexNotEqual("zlh:1",0,"zhanglonghao"));  
   tran.ListRightPushAsync("zlh:1", "zhanglonghao1");   
   bool committed = tran.Execute();
   Console.WriteLine(committed);
   Console.ReadLine();
  }

The result is false, and there is no increase or decrease data in the database. It has been consistent with the data in the figure above for a long time.

Cause analysis: 0 is zhanglonghao at this time, so listindexnotequal (“zlh: 1”, 0, “zhanglonghao”) is a false proposition. Roll back directly without executing the following insert command.

If the data is not cleared, continue with the code:


static void Main(string[] args)
  {
   var tran = db.CreateTransaction();
   tran.AddCondition(Condition.ListIndexEqual("zlh:1",0,"zhanglonghao"));  
   tran.ListRightPushAsync("zlh:1", "zhanglonghao1");   
   bool committed = tran.Execute();
   Console.WriteLine(committed);
   Console.ReadLine();
  }

The result is true, and the data result is as follows. Add a piece of data with the value of zhanglonghao1:

Cause analysis: listindexequal (“zlh: 1”, 0, “zhanglonghao”) is a true proposition. Perform the following operations to submit things.

Continue code without deleting data:


static void Main(string[] args)
  {
   var tran = db.CreateTransaction();
   tran.AddCondition(Condition.ListIndexEqual("zlh:1",0,"zhanglonghao"));  
   tran.ListRightPushAsync("zlh:1", "zhanglonghao2");
   tran.AddCondition(Condition.ListIndexNotEqual("zlh:1", 0, "zhanglonghao"));
   tran.ListRightPushAsync("zlh:1", "zhanglonghao3");
   bool committed = tran.Execute();
   Console.WriteLine(committed);
   Console.ReadLine();
  }

The result is false, and the database data has been consistent with the above for a long time.

Analysis reason: condition Listindexequal (“zlh: 1”, 0, “Zhanglong”) is true, but the following listindexnotequal (“zlh: 1”, 0, “Zhanglong”) is false. Therefore, the operation of the whole transaction is rolled back and not executed, so the database has not changed.

At this point, I won’t write redundant code, but I want to say a few points:

1. The operation to execute the command needs to be asynchronous.

2. The commands executed in the transaction will not directly see the results, so the results can not be used for judgment in the following code, because the current asynchronous command will not have any impact on the database before execute().

3. Reference documents:https://github.com/StackExchange/StackExchange.Redis/blob/master/Docs/Transactions.md

The above is the whole content of this article. I hope the content of this article can bring some help to your study or work. At the same time, I also hope to support developpaer!