The principle of golang concurrency


Mpg model

There are four important structures in go scheduler: m, P, G, sched
Mrefer toMachine, aMIt is directly associated with a kernel thread. Managed by the operating system.
P“Processor” means “processor”MThe required context is also the processor that handles user level code logic. It is responsible for bridging the scheduling context of M and G, and docking the waiting g with M.
Grefer toGoroutineIn fact, it is also a lightweight thread in essence. It includes call stack, important scheduling information, such as channel.

The number of P is determined by theGOMAXPROCSYes, it usually corresponds to the number of cores, for example, to start four threads back on a 4core server. There will be many g. each P will pop goroutine from a ready queue. In order to reduce lock contention, usually each P will be responsible for one queue

The principle of golang concurrency

The above figure shows two threads (kernel threads). OneMIt corresponds to a kernel thread, aMA context is also connectedP, a contextPEquivalent to a “processor”, a context connects one or more goroutines. In order to run goroutine, the thread must save the context

The number of context P (processor) is set to at startupGOMAXPROCSThe value of the environment variable or through the runtime functionGOMAXPROCS()。 Normally, it does not change during program execution.A fixed number of contexts means that only a fixed number of threads run go code at any time。 We can use it to adjust the call of go process to personal computer. For example, a 4-core PC runs go code on four threads.

The running P always corresponds to a kernel thread. In this way, the number of P is set to the same number of cores, which can ensure that the maximum number of P can correspond to different cores, and make full use of multi-core CPU. If the G executed by M is blocked, then p will discard the G, and m will release the P. m will hold the g. at this time, the CPU will find that the M is blocked and will send an interrupt signal. At this time, the CPU will be free to execute other M.

The purpose of the context is to let us directly release other threads when the kernel thread is blocked.

A very simple example is system callsysallA thread must not execute code and system calls at the same time. This thread M needs to give up the current context environment p so that other threads can be blockedGoroutineScheduled execution
The principle of golang concurrency
G0 in M0 executes a syscall, and then creates an M1 (which may also come from the thread cache). Then M0 discards P and waits for the return value of syscall. M1 accepts P and continues to executeGoroutineOthers in the queueGoroutine

When syscall is finished, M0 will “steal” a context. If it fails, M0 will put its gouroutine G0 into a global runqueue and put itself in the thread cache and sleep. The global runqueue is where each P pulls a new goroutine after running its own local goroutine runqueue. P also periodically checks the goroutines on the global runqueue. Otherwise, goroutines on the global runqueue may not be executed and starve to death

According to the above statement, context P will regularly check the goroutine in the global goroutine queue, so that it can do something when consuming its goroutine queue. What if the goroutine in the global goroutine queue is gone? It’s stolen from the runqueue of P in other running systems

In each PGoroutineDifferent results in different operating efficiency and time. In an environment with a lot of P and m, one P can’t run its ownGoroutineThere’s nothing to do, because maybe the other p’s have a long timegoroutineIn order to run, the queue needs to be balanced.
How to solve this problem?

Go’s method is also direct, stealing half from other p!
The principle of golang concurrency

Reference articles…

Recommended Today

Ffmpeg plays RTSP / webcam stream

This article will introduce how ffmpeg plays RTSP / webcam / file streams. The process is as follows: RTSP/Webcam/File > FFmpeg open and decode to BGR/YUV > OpenCV/OpenGL display code:…, sub module RTSP local player Ffmpeg preparation git clone cd rtsp-wasm-player export MY_ROOT=`pwd` # ffmpeg: git clone –depth 1 -b n4.4 $MY_ROOT/3rdparty/source/ffmpeg […]