Linux system basic memory management knowledge explanation

Time:2021-3-3

Memory is one of the most important resources managed by Linux kernel. Memory management system is the most important part of the operating system, because the physical memory of the system is always less than the amount of memory the system needs. Virtual memory is a strategy to overcome this contradiction. The virtual memory of the system makes the system appear to have more memory capacity than the real memory by sharing memory among processes. Linux supports virtual memory, which is to use disk as the expansion of ram, so that the available memory can be effectively expanded accordingly. The core saves the memory blocks that are not used at present to the hard disk to free up memory for other purposes. When the original content is used again, read back to memory.
 
1、 Memory usage monitoring
 
(1) Real time monitoring of memory usage
 
Use the “free” command on the command line to monitor memory usage

Copy code

The code is as follows:

#free

             total       used       free     shared    buffers     cached

Mem:        256024     192284      63740          0      10676     101004

-/+ buffers/cache:      80604     175420

Swap:       522072          0     522072

The above shows a system with 256 MB ram and 512 MB swap space. The third line of output (MEM:) shows the physical memory. The total column does not show the physical memory used by the core (usually about 1MB). The used column shows the total amount of memory used (the second row does not count buffers). The free column shows all unused memory. The shared column shows the total amount of memory shared by multiple processes. The buffers column shows the current size of the disk cache. The fifth line (swap:) for swap space, the information displayed is similar to the above. If this behavior is all zero, then no swap space is used. By default, the free command displays memory usage in kilobytes (1024 bytes). The – h parameter can be used to display memory usage in bytes, or the – M parameter can be used to display memory usage in megabytes. You can also use the – S parameter to continuously monitor the memory usage
 
#free –b –s2
 
This command will continuously report the memory usage in the terminal window and update it every 2 seconds.
 
(2) The watch and free commands are combined to monitor the memory usage in real time
 
 

Copy code

The code is as follows:

#watch -n 2 -d free

Every 2.0s: free                                        Fri Jul 6 06:06:12 2007

             total       used       free     shared    buffers     cached

Mem:        233356     218616      14740          0       5560      64784

-/+ buffers/cache:     148272      85084

Swap:       622584       6656     615928
 
 
The watch command will execute free every two seconds. Before execution, the screen will be cleared and the data will be displayed in the same position. Because the watch command does not scroll the screen, it is suitable for monitoring memory usage for a long time. You can use the – N option to control the execution frequency; you can also use the – D option to make the command display different places each time. The watch command is executed until you press [Ctrl] – [C].
 
2、 The concept of virtual memory
 
(1) Implementation mechanism of Linux virtual memory
 
The implementation of Linux virtual memory needs six mechanisms: address mapping mechanism, memory allocation recycling mechanism, cache and refresh mechanism, request page mechanism, exchange mechanism and memory sharing mechanism.
 
First of all, the memory management program maps the logical address of the user program to the physical address through the mapping mechanism. When the user program runs, if it is found that the virtual address to be used in the program does not have the corresponding physical memory, it sends out the request page request; if there is free memory to allocate, it requests to allocate the memory (so the allocation and recycling of memory are used), and the memory in use is saved Physical pages are recorded in the cache (using the caching mechanism). If there is not enough memory to allocate, the exchange mechanism is called to free up some memory. In addition, in the address mapping, we need to find the physical page through TLB (translation backup memory); in the exchange mechanism, we also need to use the exchange cache, and after the physical page content is exchanged into the exchange file, we also need to modify the page table to map the file address.
 
(2) Virtual memory capacity setting
 
You may be told that virtual memory should be allocated twice as much as physical memory, but it’s an irregular rule. If your physical storage is small, you can set it like this. If you have 1g of physical memory or more, you can shrink the virtual memory. Linux will use a lot of memory as cache, but reclaim it when resources are tight. As long as you see that swap is 0 or very small, you can rest assured, because memory is the biggest waste.

3、 Enable the vmstat command to monitor virtual memory usage
 
Vmstat is the abbreviation of virtual memory statistics, which can monitor the virtual memory, process and CPU activity of the operating system. It is the overall situation of the system statistics, the inadequacy is that a process can not be in-depth analysis. The vmstat 5 5 command is usually used for testing. You will get a summary of the data, which can reflect the real situation of the system.
 
 

Copy code

The code is as follows:

#vmstat 5 5

procs ———–memory———- —swap– —–io—- –system– —-cpu—-

 r b   swpd   free   buff cache   si   so    bi    bo   in    cs us sy id wa

 1 0 62792   3460   9116 88092    6   30   189    89 1061   569 17 28 54 2

 0 0 62792   3400   9124 88092    0    0     0    14 884   434 4 14 81 0

 0 0 62792   3400   9132 88092    0    0     0    14 877   424 4 15 81 0

 1 0 62792   3400   9140 88092    0    0     0    14 868   418 6 20 74 0

 1 0 62792   3400   9148 88092    0    0     0    15 847   400 9 25 67 0
 
The output of vmstat command is divided into six parts
 
(1) Process procs:
r: The number of processes waiting in the run queue.
b: The number of processes waiting for Io.
 
(2) Memory memory:
 
Swpd: currently available swap memory (in KB).
 
Free: free memory (in KB).
 
Buff: the amount of memory in the buffer (unit: KB).
 
Cache: the amount of memory used as cache (in KB).
 
(3) Swap Exchange page
 
Si: number of swap pages from disk to memory, unit: KB / s.
 
So: number of swap pages from memory to disk, unit: KB / s.
 
(4) IO block device:
 
Bi: the number of blocks sent to the block device, in blocks per second.
 
Bo: the number of blocks received from the block device, in blocks per second.
 
(5) System:
 
In: the number of interrupts per second, including clock interrupts.
 
CS: the number of environment (context) switches per second.
 
(6) CPU central processing unit:
CS: the time used by the user process. Expressed as a percentage.
SY: the time used by the system process. Expressed as a percentage.
ID: idle time of CPU. Expressed as a percentage.
 
If R is often greater than 4 and ID is often less than 40, it means that the CPU is heavily loaded. If Bi and Bo are not equal to 0 for a long time, it means that the physical memory capacity is too small.
 
4、 Memory leakage of Linux server and method of reclaiming memory
 
1. Definition of memory leak:
 
Generally speaking, memory leakage refers to the leakage of heap memory. Heap memory refers to the memory allocated by the program from the heap with any size (the size of the memory block can be determined during the running period of the program), and the released memory must be displayed after use. Applications generally use malloc, realloc, new and other functions to allocate a block of memory from the heap. After using it, the program must be responsible for releasing the memory block by calling free or delete. Otherwise, the memory can not be used again. We say that the memory is leaked.
 
2. The harm of memory leak
 
From the perspective of users using programs, memory leak itself will not produce any harm. As a general user, there is no memory leak at all. The real danger is the accumulation of memory leaks, which will eventually consume all the memory of the system. From this point of view, a one-time memory leak is not harmful because it does not accumulate, while an implicit memory leak is very harmful because it is more difficult to detect than frequent and accidental memory leaks. In addition to occupying more memory, the program with memory leakage will make the performance of the program decline sharply. For the server, if this happens, even if the system does not crash, it will seriously affect the use.
 
3. Detection and recovery of memory leak
 
For memory overflow and other troubles, you may encounter when writing complex programs with more pointers. In Linux or UNIX, C and C + + are the most popular tools. But our c + + program lacks the corresponding means to detect the memory information, and can only use the top instruction to observe the total dynamic memory of the process. And when the program exits, we can’t get any memory leak information.
 
Using the Kill Command
 
Using Linux commands to recycle memory, we can use PS and kill commands to detect memory usage and recycle. When using super user rights, use the command “PS”, which will list the names of all running programs and the corresponding process number (PID). The working principle of kill command is to send a system operation signal and the process number (PID) of the program to the kernel of Linux operating system.
 
Application example:
 
In order to recycle memory efficiently, you can use the command PS parameter V:
 

Copy code

The code is as follows:

 PID TTY      STAT   TIME MAJFL   TRS   DRS   RSS %MEM COMMAND

 2542 tty1     Ss+    0:00      0     8 1627   428 0.1 /sbin/mingetty tty1

 2543 tty2     Ss+    0:00      0     8 1631   428 0.1 /sbin/mingetty tty2

 2547 tty3     Ss+    0:00      0     8 1631   432 0.1 /sbin/mingetty tty3

 2548 tty4     Ss+    0:00      0     8 1627   428 0.1 /sbin/mingetty tty4

 2574 tty5     Ss+    0:00      0     8 1631   432 0.1 /sbin/mingetty tty5

 2587 tty6     Ss+    0:00      0     8 1627   424 0.1 /sbin/mingetty tty6

 2657 tty7     Ss+    1:18     12 1710 29981 7040 3.0 /usr/bin/Xorg :0 -br -a

 2670 pts/2    Ss     0:01      2   682 6213 1496 0.6 -bash

 3008 pts/4    Ss     0:00      2   682 6221 1472 0.6 /bin/bash

 3029 pts/4    S+     0:00      2    32 1783   548 0.2 ping 192.168.1.12

 3030 pts/2    R+     0:00      2    73 5134   768 0.3 ps v
 
Then, if you want to reclaim the memory of the ping command, use the following command:
 

Copy code

The code is as follows:

# Kill -9 3029

Using tool software
 
Memprof is a very attractive and easy-to-use software created by red hat’s Owen Talyor. This tool is the Boehm Demers Weiser garbage collector for the front end of gnome. This tool can be executed directly, and it works without any changes to the source code. During the execution of the program, this tool will graphically display the memory usage.

Recommended Today

Practice analysis of rust built-in trait: partialeq and EQ

Abstract:Rust uses traits in many places, from simple operator overloading to subtle features like send and sync. This article is shared from Huawei cloud community《Analysis of rust built-in trait: partialeq and EQ》Author: debugzhang Rust uses traits in many places, from simple operator overloading to subtle features like send and sync. Some traits can be automatically […]