Go program performance optimization and pprof usage

Time:2020-1-24

Performance optimization of Go program and application of pprof

The optimization of program performance is nothing more than the optimization of program resources. For servers, the two most important resources are CPU and memory. Performance optimization means that we usually require the CPU of the program to occupy as little memory as possible without affecting the data processing ability of the program. In other words, when the CPU and memory usage of the program remain the same, try to improve the data processing capacity or throughput of the program.

Go’s native tool chain provides a wealth of tools for developers to use, including pprof.

The use of pprof is divided into the following two parts.

Web program using pprof

Write a simple web service program first. The program receives requests on port 9876.


package main

import (
  "bytes"
  "io/ioutil"
  "log"
  "math/rand"
  "net/http"

  _ "net/http/pprof"
)

func main() {
  http.HandleFunc("/test", handler)
  log.Fatal(http.ListenAndServe(":9876", nil))
}

func handler(w http.ResponseWriter, r *http.Request) {
  err := r.ParseForm()
  if nil != err {
    w.Write([]byte(err.Error()))
    return
  }
  doSomeThingOne(10000)
  buff := genSomeBytes()
  b, err := ioutil.ReadAll(buff)
  if nil != err {
    w.Write([]byte(err.Error()))
    return
  }
  w.Write(b)
}

func doSomeThingOne(times int) {
  for i := 0; i < times; i++ {
    for j := 0; j < times; j++ {

    }
  }
}

func genSomeBytes() *bytes.Buffer {
  var buff bytes.Buffer
  for i := 1; i < 20000; i++ {
    buff.Write([]byte{'0' + byte(rand.Intn(10))})
  }
  return &buff
}

As you can see, we just introducednet/http/pprof , not shown.

Start the program.

We usewrk To simply simulate the request.

wrk -c 400 -t 8 -d 3m http://localhost:9876/test

At this time, we open http: / / localhost: 9876 / debug / pprof, and the following page will be displayed:

Users can click the corresponding link to browse the content. But that’s not the point, and it doesn’t seem intuitive.

Let’s open the linkhttp://localhost:9876/debug/pprof/profile Later, you can download to the file profile.

Use the pprof tool that comes with go to open it.go tool pprof test profile。 (test followed by proof is the executable compiled by the program)

inputtopThe command gets:

You can see the top 10 CPU functions, and we can optimize this analysis.

It’s just that it may not be intuitive.

Let’s type in the commandweb(you need to install graphviz in advance. You canbrew install graphviz), the interface will be opened in the browser as follows:

Can seemain.doSomeThingOne It takes up 92.46% CPU time and needs to be optimized.

The CPU time graph in the form of web is sufficient for optimization. Let’s introduce the generation of flame graph. Mac OS recommends the go torch tool. Usage andgo tool pprofBe similar.

go-torch test profile The torch.svg file is generated. You can open it with a browser, as shown in the figure.

Just now, I just talked about the generation and viewing of CPU usage analysis files. In fact, the generation of memory snapshot is similar.http://localhost:9876/debug/pprof/heap, will be downloadedheap.gzPapers.

We can also usego tool pprof test heap.gz, and enter top orweb Command to view relevant content.

General program using pprof

The Go programs we wrote are not all web programs. At this time, we can’t use the above methods.

We can still use the pprof tool, but the introduced location isruntime/pprof

Two functions are pasted here as an example:

//Generate CPU Report
func cpuProfile() {
  f, err := os.OpenFile("cpu.prof", os.O_RDWR|os.O_CREATE, 0644)
  if err != nil {
    log.Fatal(err)
  }
  defer f.Close()

  log.Println("CPU Profile started")
  pprof.StartCPUProfile(f)
  defer pprof.StopCPUProfile()

  time.Sleep(60 * time.Second)
  fmt.Println("CPU Profile stopped")
}

//Generate heap memory report
func heapProfile() {
  f, err := os.OpenFile("heap.prof", os.O_RDWR|os.O_CREATE, 0644)
  if err != nil {
    log.Fatal(err)
  }
  defer f.Close()

  time.Sleep(30 * time.Second)

  pprof.WriteHeapProfile(f)
  fmt.Println("Heap Profile generated")
}

The two functions generate cpu.profandheap.prof Papers. Still available go tool pprof Tools for analysis, not described here.

Trace Report

Direct post code:

//Generate tracking report
func traceProfile() {
  f, err := os.OpenFile("trace.out", os.O_RDWR|os.O_CREATE, 0644)
  if err != nil {
    log.Fatal(err)
  }
  defer f.Close()

  log.Println("Trace started")
  trace.Start(f)
  defer trace.Stop()

  time.Sleep(60 * time.Second)
  fmt.Println("Trace stopped")
}

Using toolsgo tool trace After analysis, a very detailed tracking report will be obtained for further program analysis and optimization. Due to the complexity of the report and the similar use method, we will not continue. Readers can try it on their own.

Post a picture on the Internet for you to have a look:

Reference: https://github.com/caibirdme/hand-to-hand-optimize-go

The above is the whole content of this article. I hope it will help you in your study, and I hope you can support developepaer more.

Recommended Today

Notes on tensorflow 2 deep learning (I) tensorflow Foundation

This series of notes records the process of learning tensorflow2, mainly based on https://github.com/dragen1860/Deep-Learning-with-TensorFlow-book Learning First of all, it needs to be clear that tensorflow is a scientific computing library for deep learning algorithm, and the internal data is stored in theTensor objectAll operations (OPS) are also based on tensor objects. data type Fundamentals in […]