Simpread golang select case implementation mechanism

Time:2020-9-26

This article is transcoded by simprad, the original address https://hitzhangjie.github.io…

Before introducing the implementation mechanism of select case, it is better to understand the Chan operation rules first, to understand when goroutine is blocked and when to wake up, which is helpful for the subsequent understanding of the implementation of select case. So, first introduce the Chan operation rules, and then introduce the implementation of select case.

1.1 Chan operation rules 1

When a goroutine wants to receive data from a non nil & non closed Chan, goroutine will first obtain the lock on the Chan, and then perform the following operations until a certain condition is met:

1) If the value buffer on Chan is not empty, it also means that the recv goroutine queue on Chan must also be empty. The received goroutine will unshift a value from the value buffer. At this time, if the send goroutine queue is not empty, because a location has been empty in the value buffer, and there is a location to write to, a send goroutine will be unshifted from the send goroutine queue, and the sending goroutine will be resumed to perform the operation of writing the data to Chan, which is actually the recovery of sending the goroutine Execute, and push the data to be sent by goroutine to the value buffer. Then, it is time to receive goroutine and continue to execute the data. In this scenario, the receiving operation of channel is called non blocking operation.

2) On the other hand, if the value buffer is empty, but the send goroutine queue is not empty, in this case, the Chan must be unbuffered Chan, otherwise the value buffer must have data. At this time, the receiving goroutine will unshift a sending goroutine from the send goroutine queue and send the goroutine The data to be sent is received (two goroutines, one with the sending data address and the other with the receiving data address, and it’s OK to copy them). Then the outgoing sending goroutine will resume execution, and the receiving goroutine can also continue to execute. In this case, the Chan receive operation is also a non blocking operation.

3) On the other hand, if both the value buffer and the send goroutine queue are empty and there is no data to receive, the received goroutine will be pushed to Chan’s recv goroutine queue. The receiving goroutine will be put into blocking state. When the recovery period is executed, wait until a goroutine tries to send data to Chan. In this scenario, the Chan receive operation is blocking operation.

1.2 Chan operation rules 2

When a goroutine common sense sends data to a non nil & non closed Chan, the goroutine will first try to obtain the lock on the Chan, and then perform the following operations until one of the conditions is met.

1) If Chan’s recv goroutine queue is not empty, in this case, the value buffer must be empty. Sending goroutine will unshift a recv goroutine from the recv goroutine queue, and then directly copy the data you want to send to the receiving address of the recv goroutine, and then resume the operation of the recv goroutine, and the current sending goroutine continues to execute. In this case, the Chan send operation is a non blocking operation.

2) If Chan’s recv goroutine queue is empty and the value buffer is dissatisfied, in this case, the send goroutine queue must be empty, because the value buffer is not satisfied with the sending goroutine, and the sending goroutine cannot be blocked. The sending goroutine pushes the data to be sent into the value buffer, and then continues to execute. In this case, the Chan send operation is a non blocking operation.

3) If Chan’s recv goroutine queue is empty and the value buffer is full, the sending goroutine will be pushed to the send goroutine queue to enter the blocking state. Wait until another goroutine attempts to receive data from Chan before waking it up and resuming execution. In this case, the Chan send operation is a blocking operation.

1.3 Chan operation rules 3

When a goroutine attempts to close a non nil & non closed Chan, the close operation will perform the following operations in turn.

1) In this case, if the data is not empty, it is necessary to continue the operation from empty to empty A recv goroutine is found in unshift. All goroutines in recv goroutine queue will be unshifted one by one and return a Val = 0 value and sendbeforeclosed = false.

2) If Chan’s send goroutine queue is not empty, all goroutines will be taken out in turn and a panic for closing a close Chan will be generated. Before this close, the data sent to Chan is still stored in Chan’s value buffer.

1.4 Chan operation rules 4

Once Chan is closed, the Chan recv operation will never be blocked, and the data written before close in Chan’s value buffer still exists. Once the data written before close in the value buffer is taken out, the subsequent receive operation will return Val = 0 and sendbeforeclosed = true.

Understanding goroutine’s blocking and non blocking operations here is helpful to understand the select case operation for Chan. The implementation mechanism of select case is described below.

If there is no default branch in the select case, you must wait until a case branch meets the conditions, and then wake up the corresponding goroutine to resume execution. Otherwise, the code will be blocked here, that is, the current goroutine will be pushed into the recv or send goroutine queue of CH corresponding to each case branch. For the same Chan, the current goroutine can be pushed to the recv or send goroutine queue of CH corresponding to each case branch Goroutine is pushed to recv and send goroutine queues at the same time.

Whether it is the ordinary Chan send, recv operation, or select Chan send, recv operation, because the goroutine blocked by Chan operation is awakened by other goroutines’ send and recv operations on Chan. We have already talked about the time when goroutine is awakened, and we need to subdivide it here.

Chan’s send and recv goroutine queues store a structure pointerSudog, member GPG points to the corresponding goroutine, elem unsafe.Pointer It points to the address of the variable to be read and written. C * hchan points to the Chan on which goroutine is blocked. If isselect is true, it means select Chan send and recv, otherwise it means Chan send and recv. g. Selectdone indicates whether the select operation is completed, that is, whether a case branch has been established.

2.1 execution logic when goroutine with Chan operation blocking wakes up

Let’s first describe the processing logic when a goroutine on Chan is awakened. If a goroutine is blocked on ch1 and CH2 because of the select Chan operation, the corresponding sudog object will be created and the corresponding pointer will be setSudog push to send and recv goroutine queues on ch1 and CH2 corresponding to each case branch, and wake them up when other coprocessors execute (select) Chan send and recv operations: 1) source filechan.goIf there is another goroutine operating on ch1, and then unshift the goroutine of ch1, take out a blocked goroutine, and execute the method when unshiftfunc (q Waitq) dequeue() sudog * *, this method returns a blocked goroutine from the waiting queue of ch1.

func (q *waitq) dequeue() *sudog {
    for {
        sgp := q.first
        if sgp == nil {
            return nil
        }
        y := sgp.next
        if y == nil {
            q.first = nil
            q.last = nil
        } else {
            y.prev = nil
            q.first = y
            sgp.next = nil // mark as removed (see dequeueSudog)
        }

        // if a goroutine was put on this queue because of a
        // select, there is a small window between the goroutine
        // being woken up by a different case and it grabbing the
        // channel locks. Once it has the lock
        // it removes itself from the queue, so we won't see it after that.
        // We use a flag in the G struct to tell us when someone
        // else has won the race to signal this goroutine but the goroutine
        // hasn't removed itself from the queue yet.
        if sgp.isSelect {
            if !atomic.Cas(&sgp.g.selectDone, 0, 1) {
                continue
            }
        }

        return sgp
    }
}

If the leader element is a previously blocked goroutine, it is detected sgp.isSelect=true To know that this is a goroutine blocked by select Chan send and recv, and then set sgp.g.selectdone to true through CAS operation to indicate that the current goroutine select operation has been completed. After that, the goroutine can be returned for use from the value buffer Read or write data to the value buffer, or directly exchange data with the goroutine that wakes it, and then the blocked goroutine can resume execution.

Setting SGP. G. selectdone to true indicates that the SGP. G has exited from the select case block that just blocked it, and the corresponding select case block can be voided. It is necessary to mention why SGP. G. selectdone is set to true? It’s over if you take the goroutine out of the team? no way! Consider the following operation on Chan. Dequeue needs to get the lock on Chan first. However, before attempting lock Chan, it is possible that Chan corresponding to multiple case branches are ready at the same time. Let’s look at an example code:

g1
go func() {
  ch1 <- 1
}()

// g2
go func() {
  ch2 <- 2
}

select {
  case <- ch1:
    doSomething()
  case <- ch2:
    doSomething()
}

Process G1 in chan.chansend Method is executed in general, prepare lock ch1, coroutine G2 also executed half, also prepare lock CH2; Coroutine G1 successfully locks ch1 to execute dequeue operation, and process G2 page successfully locks CH2 to perform DEQ queue operation. Because only one case branch can be activated in the same select case block, a member g.selectdone is added to the coroutine to identify the corresponding select case of the coroutine Whether the execution has been completed successfully (only one select case block can be processed at a certain time for a coroutine, either blocked or executed immediately). Therefore, the CAS operation is used to update the value of g.selectdone during dequeue. The successful update completes the queue operation and activates the case branch. If CAS fails, the select case is considered Other branches have been activated, the current case branch is invalid, and the select case is finished.

The CAS operation here means that when multiple branches meet the conditions, golang will randomly select one branch to execute.

2.2 how the select case block golang performs processing

source fileselect.goChinese methodselectgo(sel *hselect)To realize the processing logic of the select case block. However, due to the long length of the code, it is no longer necessary to copy and paste the code here. Those who are interested can view it by themselves. Here we only briefly describe the execution process.

Brief introduction of selectgo logic processing:

  • In the preprocessing part, each case branch is sorted according to ch address to ensure the subsequent locking in order to avoid deadlock;
  • Pass 1 deals with the judgment logic of each case branch and checks whether each case branch can satisfy the ch read-write operation immediately. If the current branch has any, it will immediately execute the ch read-write and return, and the select processing will end; if there is no branch, continue to process the next branch; if all branches are not satisfied, continue to execute the following process.
  • The Chan operation on no case branch of pass 2 can be ready immediately. The current goroutine needs to be blocked. Traverse all the case branches, respectively build the sudog corresponding to goroutine and push it to the corresponding goroutine queue of Chan corresponding to the case branch. Then gopark suspends the current goroutine and waits for the Chan operation on a branch to wake up the current goroutine. How to wake up? As mentioned earlier chan.waitq.dequeue In () method, after setting sudog.g.selectdone to 1 through CAS, the sudog is returned and resumed. In fact, this operation is used to wake up.
  • The entire select case block of pass 3 has ended its mission. The previously blocked goroutine has been awakened. Other case branches are useless and need to be discarded. Pass 3 will remove the goroutine from Chan recv and send goroutine queue corresponding to each case branch in the previously blocked select case block chan.waitq.dequeueSudog (sgp * sudog) To remove from a queue, which is a two-way linked list sudog.prev And sudog.next The time complexity of deleting sudog is O (1).

This paper briefly describes the implementation logic of select case in golang, and introduces the cooperative relationship between goroutine and Chan operations. Before that, ZMQ author Martin sustrik wrote a C-Oriented library, libmill, in imitation of golang. The actual implementation idea is similar. If you are interested, you can also look at it. Libmill source code analysis.
Simpread golang select case implementation mechanism

Simpread golang select case implementation mechanism

Recommended Today

PHP 12th week function learning record

sha1() effect sha1()Function to evaluate the value of a stringSHA-1Hash. usage sha1(string,raw) case <?php $str = “Hello”; echo sha1($str); ?> result f7ff9e8b7bb2e09b70935a5d785e0cc5d9d0abf0 sha1_file() effect sha1_file()Function calculation fileSHA-1Hash. usage sha1_file(file,raw) case <?php $filename = “test.txt”; $sha1file = sha1_file($filename); echo $sha1file; ?> result aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d similar_text() effect similar_text()Function to calculate the similarity between two strings. usage similar_text(string1,string2,percent) case […]