Week 10 of Arts | leetcode 23 merge K sorted lists | go performance tuning | project management is important

Time:2020-12-20

ARTS

Arts is an activity launched by Chen Hao in the geek time column. The aim is to keep learning through sharing.

Each person writes arts once a week: algorithm is an algorithm problem, review is to read an English article, technique / tips is to share a small technology, share is to share a point of view.

This week’s content

This week you’ll see:

  1. The golang implementation of multi-path merge sort.
  2. Go officials teach you how to do performance analysis.
  3. The project manager is really important.

Algorithm

This week’s algorithm is a combination of linked list and merge sort: 23. Merge K sorted lists

The solution of this problem is not new. For me who use go to solve this problem, the difficulty of this problem is in the realization. In gocontainer/heapI didn’t know that before, but fortunately I tried to search go for similar tools before I decided to implement a heap myself. But then again,container/heapThis tool is too simple. It only helps you to implement heaapify. Other actual data types in the heap need to be defined by yourself, and the types need to be supportedLess()Method, if not supported, you need to define it yourself. In addition, you need to implement the above types ofPop()andPush()Method to operate the underlying data. I have already written down the other points that need attention in the notes.

func mergeKLists(lists []*ListNode) *ListNode {
    if len(lists) == 0 {
        return nil
    }

    var fhead = &ListNode{}
    h := &IntHeap{}
    heap.Init(h)

    for i := range lists {
        if lists[i] == nil {
            continue
        }
        heap.Push(h, node{Val: lists[i].Val, ListIdx: i})
        lists[i] = lists[i].Next
    }

    curr := fhead
    //Container / heap will help you to do heapify automatically
    //Each time after the small top heap pop, we need to take another node from the original list
    //If the original list is empty, continue pop
    //That's why we don't deposit directly ListNode.Val  To the heap
    //In order to save the list from which the current value from the heap pop comes from, the node type is specifically declared
    //In addition, you may want to take a node from the original list of pop values
    //The reason is that the local merge logic is to put the minimum value of all the current lists in the heap at any time
    //Each time a value is taken from the original list and put into the heap. This value may not become the top of the heap after being pushed in
    //But once the next value of the original list may become the top of the heap or it may become the top of the heap before it is pushed into the heap
    //The final merge sort will be out of order
    //So the most robust and simple way is to find the next value from the original list and push it to the heap every time
    for h.Len() > 0 {
        n := heap.Pop(h).(node)
        curr.Next = &ListNode{Val: n.Val}
        if lists[n.ListIdx] != nil {
            heap.Push(h, node{
                Val:     lists[n.ListIdx].Val,
                ListIdx: n.ListIdx})
            lists[n.ListIdx] = lists[n.ListIdx].Next
        }
        curr = curr.Next
    }
    return fhead.Next
}

type node struct {
    Val     int
    ListIdx int
}

type IntHeap []node

func (h IntHeap) Len() int { return len(h) }

//Less increment means small top heap
func (h IntHeap) Less(i, j int) bool { return h[i].Val < h[j].Val }

func (h IntHeap) Swap(i, j int) { h[i], h[j] = h[j], h[i] }

func (h *IntHeap) Push(x interface{}) {
    *h = append(*h, x.(node))
}

func (h *IntHeap) Pop() interface{} {
    ret := (*h)[len(*h)-1]
    *h = (*h)[:len(*h)-1]
    return ret
}

Review article recommendation

This week’s recommended article is a popular science article on the go official website: profiling Go programs

This article is actually an official response to the following paper loop recognition in C + + / Java / go / Scala, because the test results of several languages in the paper are not “friendly” to golang. All in all, from the results, go did not perform well in the above benchmark tests. Go officials said that “it’s not that our performance is not very good, but you are not interoperable! “

The Go program presented in that paper runs quite slowly, making it an excellent opportunity to demonstrate how to use Go’s profiling tools to take a slow program and make it faster.

How about it? Do you feel familiar?

So the official how to use itpprofTools, and then to the specific optimization of go code CPU usage and memory efficiency, are introduced in this article. Of course, for the specific optimization content can not be all inclusive. This paper mainly focuses on reducing the CPU utilization rate for some data types (such as changing map to slice) and making local cache for some frequently created and deleted objects to reduce CPU consumption caused by memory application and garbage collection.

The above two measures are effective from the benchmark results, and then the author does not forget to ridicule the previous paper.

Having a garbage-collected language doesn’t mean you can ignore memory allocation issues. In this case, a simple solution is to introduce a cache so that each call to FindLoops reuses the previous call’s storage when possible. (In fact, in Hundt’s paper, he explains that the Java program needed just this change to get anything like reasonable performance, but he did not make the same change in the other garbage-collected implementations.)

Finally, after a meal of operation, the author gives the optimization result: it can be at least 6 to 11 times faster than the conclusion in the paper. At the same time, the author emphasizes that there are still places that can be optimized. However, there is no difference between the tools and methods used in the optimization process.

There’s more we can do to clean up the program and make it faster, but none of it requires profiling techniques that we haven’t already shown.

Finally, the author would like to add that go and C + + can be “to some extent” comparable.

Benchmarks are only as good as the programs they measure. We used go tool pprof to study an inefficient Go program and then to improve its performance by an order of magnitude and to reduce its memory usage by a factor of 3.7. A subsequent comparison with an equivalently optimized C++ program shows that Go can be competitive with C++ when programmers are careful about how much garbage is generated by inner loops

Tip programming skills

I wanted to find a transformation technique this week, but I failed. Tears dried up.

Share flash

On Friday night, I chatted with an old classmate about my experiences in different company project management modes. For my current job, I don’t need to consider the progress of the project at all, because I have experience in recommending the whole project. I just need to be responsible for my own part and make sure it’s done on time. If the development progress of other departments or teams is slower than expected, I will ask the project manager to solve the problem. However, the company where my classmate works seems not to have strict division of labor in this respect. As a result, my classmate did a lot of work to speed up the project progress of the docking team, and even carried out some friendly assistance type development work for the other party’s team.

However, the project was not delivered on time in the end. On the contrary, the team leader of the other side felt that my classmate’s work content was not saturated enough, and he had always been a “king of the mouth”. In fact, he had completed his own development work half a month ahead of schedule. This kind of thing is impossible for me to happen. I can’t go to other teams to speed up their work in the first place. Because that’s not my responsibility. Moreover, because I don’t know the other party’s project schedule and priorities, I’ll never get involved in other teams.

To be honest, I was shocked to hear about my classmate’s experience. First of all, I can’t understand why he interferes with the work of other teams. Thirdly, I can’t understand the behavior of the other team. These should have been done by the project manager, if the normal development is smooth, it is difficult to notice. But once there are some problems that cause the project to be pushed forward artificially, the importance of the project manager will be highlighted.

So in the normal development, don’t always think that the project manager is urging you and me to work. Most of the time, they are helping you solve many problems that you absolutely don’t want to solve by yourself.

Reading list of the week

  • Geek time MySQL actual combat 45
    Row lock
  • Learn raft protocol through animation
    Popular! Easy to understand! Also easy to forget, ha ha ha.

Recommended Today

Notes of C + + primer Chapter 8 IO Library

Iostream defines the basic types for reading and writing streams, fsstream defines the types for reading and writing named files, and ssstream defines the types for reading and writing memory string objects. The standard library enables us to ignore the differences between these different types of flows, which is implemented through the inheritance mechanism. Because […]