Go goroutine and channel

Time:2020-4-4

Goroutine and channel

Goroutine – look at a requirement
Demand: which of the 1-9000000000 numbers are prime numbers?

Analysis ideas:
1) The traditional method is to use a loop to judge whether each number is prime or not. [very slow]
2) Use concurrent or parallel methods to assign the task of counting prime numbers to multiple goroutines to complete. Then goroutine will be used

Goroutine basic introduction

Introduction to process and thread

Program, process, and thread relationships

Concurrent and parallel

Concurrent and parallel
1) Multithreaded programs running on a single core are concurrent
2) Multithreaded programs run on multiple cores, that is, parallel

Go orchestration and go main thread

Go main thread (some programmers call it thread directly / can also be understood as process): on a go thread, there can be multiple coroutines. You can understand that a coroutine is a lightweight thread [compiler optimizes].

Characteristics of go process
1) Independent stack space
2) Shared program heap space
3) Scheduling is controlled by the user
4) Coroutines are lightweight threads

Goroutine – quick start

Case description
Please write a program to complete the following functions:
1) In the main thread (which can be understood as a process), start a goroutine, and the process outputs “Hello, world” every 1 second
2) Output “Hello, golang” every other second in the main thread, and exit the program after outputting 10 times
3) The main thread and goroutine are required to execute at the same time

package main
import (
    "fmt"
    "strconv"
    "time"
)

//In the main thread (which can be understood as a process), start a goroutine, which outputs "Hello, world" every 1 second
//Output "Hello, golang" every other second in the main thread, and exit the program after outputting 10 times
//The main thread and goroutine are required to execute at the same time

//Write a function to output "Hello, world" every 1 second
func test() {
    for i := 1; i <= 10; i++ {
        fmt.Println("tesst () hello,world " + strconv.Itoa(i))
        time.Sleep(time.Second)
    }
}

func main() {

    Go test() // a collaboration is started

    for i := 1; i <= 10; i++ {
        fmt.Println(" main() hello,golang" + strconv.Itoa(i))
        time.Sleep(time.Second)
    }
}

Quick start summary

1) The main thread is a physical thread that directly acts on the CPU. It’s heavyweight and very CPU intensive.
2) The coroutine is opened from the main thread. It is a lightweight thread and a logical state. The consumption of resources is relatively small.
3) The cooperative mechanism of golang is an important feature, which can easily open tens of thousands of cooperative processes. The concurrency mechanism of other programming languages is generally based on threads, with too many threads open and large resource consumption. This highlights the advantages of golang in concurrency

Goroutine’s scheduling model

Basic introduction of MPG mode

Explain what mpg means:
M (machine): the main thread of the operating system
P (processor): the resource (context) needed for the execution of the cooperation process can be regarded as a local scheduler, which makes the go code run on a thread. It is the key to realize the mapping from n:1 to N: M
Gorountine: CO program, with its own stack. Contains instruction pointer and other information (waiting channels, etc.) for scheduling. There can be multiple GS under a p

State I of MPG mode operation

  • The number of P can be set through gomaxprocs(), which actually represents the true degree of concurrency, that is, how many goroutines can run at the same time. P also maintains the queue of G (the process queue called runqueue). Every time a statement of m in the go code is executed, P adds a G (taken from the runqueue queue) at the end, and at the next scheduling point (P), G is taken from the runqueue queue.

  • P can go to another OS thread (m) when the OS thread (main thread, or m) is blocked! The scheduler in go ensures that there are enough threads to run all the P’s. When G0 in M0 is enabled to be syscall, the P under M0 is transferred to another thread M1 (either created or existing). M1 accepts P (including G in all statuses in the queue of runqueue brought by P, but excluding G0 already called by syscall), and continues to run. M0 will wait for the return value of G0 of syscall. When the syscall of G0 is finished, his main thread M0 will try to get a p to run G0. Generally, he will steal a P from other m, if not, he will put G0 into a global runqueue, and then put himself (M0) into the thread pool or turn to sleep.

Set the number of CPUs that golang runs

Introduction: in order to make full use of the advantages of multi CPU, in the golang program, set the number of running CPUs

package main
import (
    "runtime"
    "fmt"
)

func main() {
    cpuNum := runtime.NumCPU()
    fmt.Println("cpuNum=", cpuNum)

    //You can set up and use multiple CPUs by yourself
    runtime.GOMAXPROCS(cpuNum - 1)
    fmt.Println("ok")
}

Channel – a need

Requirement: now we need to calculate the factorial of each number from 1 to 200, and put the factorial of each number into the map. Finally, it shows.
Goroutine required

Analysis ideas:
1) Goroutine is efficient, but there will be concurrent / parallel security problems
2) Here, the problem of how different goroutines communicate is raised

code implementation
1) Use goroutine to complete (see what happens when using goroutine for concurrent completion? Then solve it)
2) When running a program, how to know if there is a resource competition problem. The method is very simple. When compiling the program, add a parameter race

How to communicate between different goroutines

1) Mutex of global variables
2) Use pipeline channel to solve

Using global variable to lock synchronization

Because the global variable m is not locked, there will be a problem of resource contention, and an error will appear in the code, prompting concurrent map
writes
Solution: add mutex
The factorial of our number is very large, and the result will be out of bounds. We can change the factorial to sum + = Uint64 (I)

package main
import (
    "fmt"
    _ "time"
    "sync"
)

//Requirement: now we need to calculate the factorial of each number from 1 to 200, and put the factorial of each number into the map.
//Finally, it shows. Goroutine required 

// train of thought
//1. Write a function to calculate the factorial of each number and put it into the map
//2. We start multiple processes and statistically put the results into the map
//3. Map should make a global

var (
    myMap = make(map[int]int, 10)  
    //Declare a global mutex
    //Lock is a global mutex, 
    //Sync is package: synchronized synchronization
    //Mutex: mutual exclusion
    lock sync.Mutex
)

//The test function is to calculate n! And put the result into mymap
func test(n int) {
    
    res := 1
    for i := 1; i <= n; i++ {
        res *= i
    }

    //Here we put res into mymap
    // lock
    lock.Lock()
    myMap[n] = res //concurrent map writes?
    // unlock
    lock.Unlock()
}

func main() {

    //Let's start multiple cooperation programs to complete this task [200]
    for i := 1; i <= 20; i++ {
        go test(i)
    }


    //Sleep for 10 seconds [second question]
    //time.Sleep(time.Second * 5)

    //Here we output the result, the variable
    lock.Lock()
    for i, v := range myMap {
        fmt.Printf("map[%d]=%d\n", i, v)
    }
    lock.Unlock()

}

Why channel is needed

1) Previously, global variable lock synchronization was used to solve goroutine communication, but it was not perfect
2) It is difficult to determine the time when the main thread is waiting for all goroutines to complete. We set 10 seconds here, just for estimation.
3) If the main thread sleeps for a long time, the waiting time will be longer. If the waiting time is shorter, goroutine may still be working
State, which will be destroyed with the exit of the main thread
4) The communication is realized by locking and synchronizing the global variables, and the reading and writing operations of global variables are not realized by using multiple cooperation procedures.
5) All the above analyses are calling for a new communication mechanism channel

Basic introduction to channel

1) Channle is essentially a data structure queue
2) Data is first in first out
3) Thread safety. When accessing multiple goroutines, there is no need to lock them. That is to say, the channel itself is thread safe
4) There are types of channels. A string channel can only store string type data.

Define / declare channel

Var variable name Chan data type
Give an example:

Var int Chan Chan int (intchan for int data)
Var map Chan Chan map [int] string (mapchan for map [int] string type)
var perChan chan Person
var perChan2 chan *Person
...

Explain
Channel is a reference type
Channel must be initialized to write data, i.e. make before using
There are types of pipes. Intchan can only write int

package main
import (
    "fmt"
)

func main() {

    //Demonstrate the use of pipes
    //1. Create a pipe that can hold 3 int types
    var intChan chan int
    intChan = make(chan int, 3)

    //2. See what intchan is
    Fmt.printf ("value of intchan =% v address of intchan itself =% P \ n", intchan, & intchan)


    //3. Write data to pipeline
    intChan

Initialization of pipeline, writing data to pipeline, reading data from pipeline and basic precautions

Notes for channel use

1) Only the specified data type can be stored in the channel
2) Once the data of channle is full, it can’t be put in any more
3) If the data is removed from the channel, you can continue to put
4) If the channel data has been retrieved and retrieved again without using the protocol, the message “dead lock” will be reported

Read and write channel case demonstration

package main
import (
    "fmt"
)

type Cat struct {
    Name string
    Age int
}

func main() {

    //Define a pipeline for storing 3 data of any data type
    //var allChan chan interface{}
    allChan := make(chan interface{}, 3)

    allChan

Traverse and close of channel

Channel shutdown

Use the built-in function close to close the channel. When the channel is closed, you can no longer write data to the channel, but you can still read data from the channel

Traversal of channels

Channel supports for — range traversal. Please pay attention to two details
1) During traversal, if the channel is not closed, a deadlock error will occur
2) During traversal, if the channel has been closed, the data will be traversed normally. After traversal, the traversal will exit.

A case study of channel traversal and closure

package main
import (
   "fmt"
)

func main() {

   intChan := make(chan int, 3)
   intChan

Application examples — channel and goroutine

package main
import (
    "fmt"
    "time"
)


//write Data
func writeData(intChan chan int) {
    for i := 1; i <= 50; i++ {
        //Put in data
        intChan

Application example 2 – blocking

If the above code, comment outgo readData(intChan, exitChan)What happens? Because the pipeline has length, when the compiler finds that a pipeline has only write but no read, the pipeline will block (it doesn’t matter if the frequency of read and write is inconsistent)!

Application example 3

Demand:
Which of the 1-200000 numbers required to be counted are prime numbers? This problem was raised at the beginning of this chapter. Now that we have the knowledge of goroutine and channel, we can complete [test data: 80000]

Analysis ideas:

The traditional method is to use a loop to determine whether each number is a prime [OK].
In the way of concurrent / parallel, the task of counting prime number is assigned to several (4) goroutines to complete, and the completion time is short.

Traditional method, a process

package main
import (
    "time"
    "fmt"
)

func main() {

        start := time.Now().Unix()
        for num := 1; num <= 80000; num++ {

            Flag: = true // assuming prime
            //Judge whether num is prime
            for i := 2; i < num; i++ {
                If num% I = = 0 {// indicates that the num is not a prime number
                    flag = false
                    break
                }
            }

            if flag {
                //Put this number in primechan
                //primeChan

Four projects

package main
import (
    "fmt"
    "time"
)



//Put 1-8000 pieces into intchan
func putNum(intChan chan int) {

    for i := 1; i <= 80000; i++ {    
        intChan

Conclusion: after using go process, the execution speed is at least 4 times faster than that of the common method in theory (twice as fast as mine)

Channel use details and precautions

1) Channel can be declared read-only or write only [case demonstration]

package main
import (
    "fmt"
)

func main() {
    //Pipes can be declared read-only or write only

    //1. By default, pipes are bidirectional
    //Var chan1 Chan int // readable and writable
    
    //2 declared write only
    var chan2 chan

3) Using select can solve the blocking problem of fetching data from the pipeline

package main
import (
    "fmt"
    "time"
)

func main() {

    //Using select can solve the blocking problem of fetching data from the pipeline

    //1. Define 10 data ints of a pipeline
    intChan := make(chan int, 10)
    for i := 0; i < 10; i++ {
        intChan

4) Recover is used in goroutine to solve the problem of panic in the cooperation process, which leads to program crash

If we start a cooperation process, but there is panic in the cooperation process, the whole program will crash. At this time, we can use recover to capture panic in goroutine, so that if there is a problem in the cooperation process, the main thread will not be affected

package main
import (
    "fmt"
    "time"
)

// function
func sayHello() {
    for i := 0; i < 10; i++ {
        time.Sleep(time.Second)
        fmt.Println("hello,world")
    }
}
// function
func test() {
    //Here we can use defer + recover
    defer func() {
        //Capture the panic thrown by test
        if err := recover(); err != nil {
            FMT. Println ("test() error", ERR)
        }
    }()
    //A map is defined
    var myMap map[int]string
    myMap[0] = "golang" //error
}

func main() {

    go sayHello()
    go test()


    for i := 0; i < 10; i++ {
        fmt.Println("main() ok=", i)
        time.Sleep(time.Second)
    }

}

Recommended Today

Python basics Chinese series tutorial · translation completed

Original: Python basics Python tutorial Protocol: CC by-nc-sa 4.0 Welcome anyone to participate and improve: a person can go very fast, but a group of people can go further. Online reading Apache CN learning resources catalog introduce Seven reasons to learn Python Why Python is great Learn Python introduction Executing Python scripts variable character string […]