An analysis of unmanaged memory leak in an API of. Net

Time:2021-10-19

1: Background

1. Tell a story

At the end of July, a friend found me on Wx and said that his program memory occupied 8g, while the hosting only occupied 1.5g. He asked where the rest of the memory went? The screenshot is as follows:

From the help content, this friend is really very polite. He talks about money all the time. It really hurts his feelings. If a friend has been paying attention to my sharing, he should know that I have always analyzed dump for free. Of course, my knowledge and experience also have boundaries. I can’t decide some dump, but I still try my best to find the answer.

Here, I need to talk about the workplace. In my subconscious mind or in my team, of course, these difficult problems are solved by the technical leaders, but I found that this is not the case in several cases. The technical manager can’t subcontract, so let him hire another Gaoming… Is there a big man who can analyze it.

Well, don’t talk much. It’s urgent to talk to WinDbg.

2: WinDbg analysis

1. Is it really an unmanaged leak?

I have mentioned in many articles on analyzing memory leaks. First, use dichotomy to determine which part of the memory leak is (managed or unmanaged).

0:000> !address -summary

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                    387     7df2`11ac1000 ( 125.946 TB)           98.39%
                              2229      20c`a21bb000 (   2.049 TB)  99.75%    1.60%
Heap                                   1081        1`33914000 (   4.806 GB)   0.23%    0.00%
Image                                  1674        0`0e4be000 ( 228.742 MB)   0.01%    0.00%
Stack                                   973        0`0a140000 ( 161.250 MB)   0.01%    0.00%
TEB                                     324        0`00288000 (   2.531 MB)   0.00%    0.00%
Other                                    11        0`001d9000 (   1.848 MB)   0.00%    0.00%
PEB                                       1        0`00001000 (   4.000 kB)   0.00%    0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_MAPPED                              300      200`00f9e000 (   2.000 TB)  97.35%    1.56%
MEM_PRIVATE                            3869        d`dd7ed000 (  55.461 GB)   2.64%    0.04%
MEM_IMAGE                              2124        0`0fda4000 ( 253.641 MB)   0.01%    0.00%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE                                387     7df2`11ac1000 ( 125.946 TB)           98.39%
MEM_RESERVE                            1763      20b`d9903000 (   2.046 TB)  99.60%    1.60%
MEM_COMMIT                             4530        2`14c2c000 (   8.324 GB)   0.40%    0.01%


0:000> !eeheap -gc
Number of GC Heaps: 40
------------------------------
Heap Size:               Size: 0x3322e60 (53620320) bytes.
------------------------------
GC Heap Size:            Size: 0x603046b0 (1613776560) bytes.

from!address -summaryand!eeheap -gcTwo orders, as my friend said:MEM_COMMIT=8.3G, GC Heap=1.5G, I’ll go. It’s really a difficult unmanaged memory leak. Since it’s in hell mode, you should continue to watch it. If you want to continue troubleshooting, you must first watch itwindows ntPile.

2. View windows NT heap

In fact, whether managed c# or unmanaged C, C + +, they need to call windows to allocate memoryVirtualAlloc,HeapAllocAPI to Windows NT. The next research direction is how to find these NT heaps that can not be seen by. Net. You can use WinDbg!heap -sCommand.

0:000> !heap -s


************************************************************************************************************************
                                              NT HEAP STATS BELOW
************************************************************************************************************************
LFH Key                   : 0x0e4dcfd61ab09dd9
Termination on corruption : ENABLED
          Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 
                            (k)     (k)    (k)     (k) length      blocks cont. heap 
-------------------------------------------------------------------------------------
000001bacd190000 00000002 4841944 4810424 4840388  13556  2042   303    2    dd8   LFH
000001baccfb0000 00008000      64      4     64      2     1     1    0      0      
000001bacd4d0000 00001002    8772   6748   7216   1045   191     4    0     38   LFH
    External fragmentation  15 % (191 free blocks)
000001bacdf90000 00001002    2636    404   1080     33     3     2    0      0   LFH
000001bace620000 00001002    8772   4052   7216   3874    13     7    0     1f   LFH
    External fragmentation  95 % (13 free blocks)
000001bace610000 00001003      60      8     60      6     1     1    0    N/A   
000001bace540000 00001002    1616     24     60      4     2     1    0      1   LFH
000001baceb50000 00001002    4680   1228   3124    504    99     3    0      0   LFH
    External fragmentation  41 % (99 free blocks)
000001baceb20000 00041002      60      8     60      5     1     1    0      0      
000001baceb10000 00041002    1616     68     60      4     3     1    0      0   LFH
000001c7738a0000 00001002   49336  19316  47780   8249    43    22    0    13b   LFH
    External fragmentation  42 % (43 free blocks)
000001c7753c0000 00001002   13712   8460  12156    968    29     6    0     1c   LFH
    External fragmentation  11 % (29 free blocks)
000001c7763f0000 00001002    8772   3944   7216    423    25     4    0     3f   LFH
000001ba977c0000 00001002    1080    376   1080    365     3     2    0      0      
-------------------------------------------------------------------------------------

As can be seen from the above information, there are currently 14 heaps, of which the largest one accounts for4.8G, why is this heap so big? Next, take a detailed look at this heap, which can be used!ext.heap -stat -h 000001bacd190000

0:000> !ext.heap -stat -h 000001bacd190000
 heap @ 000001bacd190000
group-by: TOTSIZE max-display: 20
    size     #blocks     total     ( %) (percent of total busy bytes)
    20034 8eee - 11df90858  (96.44)
    2ee0000 2 - 5dc0000  (1.98)
    851 1c2b - ea419b  (0.31)
    2ac00 28 - 6ae000  (0.14)
    27d8 268 - 5fdfc0  (0.13)
    24000 28 - 5a0000  (0.12)
    d51 564 - 47c8a4  (0.09)
    10d1 3e7 - 419f97  (0.09)
    fd1 415 - 409025  (0.09)
    29d1 12f - 317e5f  (0.07)
    138 18b0 - 1e1680  (0.04)
    12c 188b - 1cc2e4  (0.04)
    1000 17e - 17e000  (0.03)
    2000 8e - 11c000  (0.02)
    200 899 - 113200  (0.02)
    ad1 178 - fe2f8  (0.02)
    478 367 - f3448  (0.02)
    7c8 1b9 - d6788  (0.02)
    1c038 7 - c4188  (0.02)
    f520 c - b7d80  (0.02)

Many people may not understand the above divinatory symbols. First of allbusyIt indicates those that have not been released recently. From the divinatory head, there are 36590 blocks with size = 20034, with a total occupation of:11df90858 = 4797827160byte = 4.7G, the next question is obvious. What are these blocks??? To find the answer, put this3wMany block information are displayed. You can use the command:!ext.heap -flt s 20034

0:000> !ext.heap -flt s 20034
    _HEAP @ 1bacd190000
              HEAP_ENTRY Size Prev Flags            UserPtr UserSize - state
        000001c771f2ad30 2004 0000  [00]   000001c771f2ad40    20034 - (busy)
        000001c774a65160 2004 2004  [00]   000001c774a65170    20034 - (busy)
        000001c774a851a0 2004 2004  [00]   000001c774a851b0    20034 - (busy)
        000001c774aa51e0 2004 2004  [00]   000001c774aa51f0    20034 - (busy)
        000001c774ac5220 2004 2004  [00]   000001c774ac5230    20034 - (busy)
        000001c774ae5260 2004 2004  [00]   000001c774ae5270    20034 - (busy)
        000001c774b052a0 2004 2004  [00]   000001c774b052b0    20034 - (busy)
        000001c774b29320 2004 2004  [00]   000001c774b29330    20034 - (busy)
        000001c774b49360 2004 2004  [00]   000001c774b49370    20034 - (busy)
        000001c774b693a0 2004 2004  [00]   000001c774b693b0    20034 - (busy)
        000001c774b893e0 2004 2004  [00]   000001c774b893f0    20034 - (busy)
          unknown!noop
        000001c774ba9420 2004 2004  [00]   000001c774ba9430    20034 - (busy)
        ...
        ...

There are too many block information, so I’ll post a part here. The heap listed above_ Entry is the first address of the block, and then I passdcAfter a meal, I found a lot of the following output.

0:000> dc 000001c774a65160 L 50
000001c7`74a65160  dddddddd 00000000 cada2944 0c85cc02  ........D)......
000001c7`74a65170  74a21070 000001c7 74a851b0 000001c7  p..t.....Q.t....
000001c7`74a65180  00000000 00000000 00000000 00000001  ................
000001c7`74a65190  00020000 00000000 0000007a fdfdfdfd  ........z.......
000001c7`74a651a0  00c801aa 55028000 05040355 44b60706  .......UU......D
000001c7`74a651b0  693d6f55 69502c31 32693d6e 7361502c  Uo=i1,Pin=i2,Pas
000001c7`74a651c0  726f7773 33733d64 6f72472c 693d7075  sword=s3,Group=i
000001c7`74a651d0  74532c34 54747261 3d656d69 452c3569  4,StartTime=i5,E
000001c7`74a651e0  6954646e 693d656d 75532c36 41726570  ndTime=i6,SuperA
000001c7`74a651f0  6f687475 657a6972 0a37693d 72657375  uthorize=i7.user
000001c7`74a65200  68747561 7a69726f 2c323d65 3d6e6950  authorize=2,Pin=
000001c7`74a65210  412c3169 6f687475 657a6972 656d6954  i1,AuthorizeTime
000001c7`74a65220  656e6f7a 693d6449 75412c32 726f6874  zoneId=i2,Author
000001c7`74a65230  44657a69 49726f6f 33693d64 6c6f680a  izeDoorId=i3.hol
000001c7`74a65240  79616469 482c333d 64696c6f 693d7961  iday=3,Holiday=i
000001c7`74a65250  6f482c31 6164696c 70795479 32693d65  1,HolidayType=i2
000001c7`74a65260  6f6f4c2c 33693d70 6d69740a 6e6f7a65  ,Loop=i3.timezon
000001c7`74a65270  2c343d65 656d6954 656e6f7a 693d6449  e=4,TimezoneId=i
000001c7`74a65280  75532c31 6d69546e 693d3165 75532c32  1,SunTime1=i2,Su
000001c7`74a65290  6d69546e 693d3265 75532c33 6d69546e  nTime2=i3,SunTim

To tell you the truth, it’s really tiring to use DC one by one. Here I’ll write a simple script to DC out the first 1W blocks to see what the content is like?

"use strict";

var index = 1;

function initializeScript() { return [new host.apiVersionSupport(1, 7)]; }
function log(str) { host.diagnostics.debugLog(str + "\n"); }
function exec(str) { log("\n" + str); return host.namespace.Debugger.Utility.Control.ExecuteCommand(str); }

function invokeScript() {
    show_heap_s();
}

function show_heap_s() {

    //get top 1 
    var output = exec("!heap -s").Skip(10).First();

    var h_address = output.split(' ')[0];

    show_max_blocksize(h_address);
}

function show_max_blocksize(address) {

    var output = exec("!ext.heap -stat -h " + address).Skip(3).First();

    var block_size = output.trim().split(' ')[0];

    show_all_blocksize(block_size);
}

function show_all_blocksize(blocksize) {

    var output = exec("!ext.heap -flt s " + blocksize).Take(10000);
    for (var line of output) {

        var heap_entry_address = line.trim().split(' ')[0];

        if (heap_entry_address.indexOf("00") == -1) continue;

        show_heap_entry(heap_entry_address);
    }
}

function show_heap_entry(heap_entry_address) {

    var pageIndex = (index++);

    var path = ".writemem D:\\dumps\\winform-memory-leak\\file\\" + pageIndex + ".txt " + heap_entry_address + " L?0x500";

    var output = exec(path);

    log("pageIndex=" + pageIndex);
}

After the script is executed, the output results are as follows:

I asked my friend what these strings are about? Why so many strings in the unmanaged have not been released? My friend told me that this is probably the access control related business, which interacts with c# through PLC. After analysis, all the information I can provide has been provided. Next, I need to confirm with the access control business party how to further locate and improve it.

3: Summary

It seems that this is the first time in the 20 dump case sharing to talk about the problem of unmanaged memory leakage. I once said on site B that I only focus on the analysis of. Net managed memory leakage, which seems difficult to realize. Indeed, there are countless examples of unmanaged memory leakage caused by the interaction between c# and Lua, C + +, COM and embedded browsers

More high quality dry goods: see my GitHub:dotnetfly

图片名称