Author: Darren_ Wen
This paper introduces the principle and operation of Linux memory mechanism, virtual memory swap, buffer / cache release and so on.
1、 What is the memory mechanism of Linux?
We know that reading and writing data directly from physical memory is much faster than reading and writing data from hard disk. Therefore, we hope that all data reading and writing are completed in memory, while memory is limited. Thus, the concepts of physical memory and virtual memory are introduced.
Physical memory is the memory size provided by the system hardware, which is the real memory. Compared with physical memory, there is a concept of virtual memory in Linux. Virtual memory is a strategy proposed to meet the shortage of physical memory. It is a piece of logical memory that is virtualized by using disk space. The disk space used as virtual memory is called swap space.
As an extension of the physical memory, Linux will use the virtual memory of the swap partition when the physical memory is insufficient. To be more specific, the kernel will write the temporarily unused memory block information to the swap space. In this way, the physical memory is released, and this memory can be used for other purposes. When the original content is needed, the information will be exchanged again Read into physical memory.
In order to make full use of the physical memory, the kernel will automatically exchange the data blocks that are not frequently used in the physical memory to the virtual memory, and keep the frequently used information to the physical memory.
In order to deeply understand the operating mechanism of Linux memory, we need to know the following aspects:
Linux system will exchange pages from time to time to keep as much free physical memory as possible. Even if there is no need for memory, Linux will exchange out temporarily unused memory pages. This avoids the time required to wait for the exchange.
It is conditional for Linux to switch pages. Not all pages are switched to virtual memory when they are not in use. According to the “most frequently used” algorithm recently, the Linux kernel only switches some rarely used page files to virtual memory. Sometimes we can see such a phenomenon: there is still a lot of physical memory in Linux, but a lot of swap space is also used. In fact, this is not surprising. For example, when a process that takes up a lot of memory resources runs, it needs to consume a lot of memory resources. At this time, some rarely used page files will be exchanged into virtual memory. However, when the process that takes up a lot of memory resources ends and releases a lot of memory, the pages that have just been swapped out will be released Files will not be automatically exchanged into the physical memory. Unless this is necessary, then the physical memory of the system will be idle a lot at the moment, and the swap space is also being used. The phenomenon just mentioned appears. Don’t worry about this, just know what’s going on.
The pages in the swap space will be first switched to the physical memory. If there is not enough physical memory to hold these pages, they will be exchanged out immediately. As a result, there may not be enough space in the virtual memory to store these swapped pages, which will eventually lead to false crashes and service exceptions in Linux However, it can recover itself within a period of time, but the recovered system is basically unavailable.
Therefore, it is very important to plan and design the usage of Linux memory
In Linux operating system, when the application program needs to read the data in the file, the operating system first allocates some memory, reads the data from the disk into the memory, and then distributes the data to the application program; when it needs to write data to the file, the operating system allocates the memory to receive the user data, and then writes the data from the memory to the disk. However, if there is a large amount of data that needs to be read from the disk to the memory or written from the memory to the disk, the read-write performance of the system becomes very low, because whether reading data from the disk or writing data to the disk is a very time-consuming and resource consuming process. In this case, Linux introduces buffers and cached mechanisms.
Both buffers and cached are memory operations, which are used to save files that have been opened by the system and file attribute information. When the operating system needs to read some files, it will first use buffers If it is found, it will be read directly to the application program. If it is not found, it will be read from the disk. This is the caching mechanism of the operating system. Through caching, the performance of the operating system is greatly improved. But the contents of buffers and cached buffers are different.
Buffers are used for buffer block devices. It only records the metadata of the file system and tracking in flight pages, while cached is used to buffer files. More popular: buffers are mainly used to store what is in the directory, file attributes and permissions, etc. Cached is directly used to remember the files and programs we have opened.
In order to verify whether our conclusion is correct, you can open a very large file through VI to see the changes of cached, and then VI this file again to feel the difference between the two opening speeds. Is the speed of opening the second time significantly faster than that of the first? Then execute the following command:
Find / – name. Conf to see if the value of buffers changes, and then repeat the find command to see how the display speed is different twice.
2、 When did Linux start to use virtual memory (SWAP)?
[[email protected] ~]# cat /proc/sys/vm/swappiness60
The above 60 indicates that swap will be used only when 40% of the physical memory is used (refer to network resources: when the remaining physical memory is less than 40% (40 = 100-60), the swap space will be used when the remaining physical memory is less than 40%) Swap space, when the swippiness = 100, indicates active use of swap partition, and timely transfer the data in memory to the swap space.
The higher the value, the more inclined to use swap. It can be set to 0, which does not prohibit the use of swap, but only minimizes the possibility of using swap.
In general, the recommended partition setting for swap is twice the memory (when the memory is less than or equal to 4G). If the memory is greater than 4G, swap only needs to be larger than the memory. In addition, try to lower the swappiness, so that the performance of the system will be better.
B. Modify the swappiness parameter
#Temporary modification:[ [email protected] ~]# sysctl vm.swappiness=10 vm.swappiness = 10 [[email protected] ~]# cat /proc/sys/vm/swappiness 10 #Permanent modification: [[email protected] ~]# vim /etc/sysctl.conf Add parameters: vm.swappiness = 35 Then directly: [[email protected] ~]# sysctl -p /etc/sysctl.conf #Check whether it works: cat / proc / sys / VM / swappiness35
It takes effect immediately, and a restart can also take effect.
3、 How to free memory?
General system is the key configuration file / proc / sys / VM / drop, which will not automatically release memory_ caches。 This file records the parameters of cache release. The default value is 0, that is, cache will not be released. Its value can be any number between 0 and 3, representing different meanings
- 0 – no release
- 1 – release page cache
- 2 – release dentries and inodes
- 3 – release all caches
Obviously, there is a lot of free memory
4、 How to release swap?
Premise: first of all, make sure that the remaining memory is greater than or equal to the usage of swap, otherwise it will be down! According to the memory mechanism, once the swap partition is released, all the files stored in the swap partition will be transferred to the physical memory. The release of swap is usually accomplished by remounting the swap partition.
a. Check where the current swap partition is mounted? b. Shut down the partition C. check the status: D. check whether the swap partition is shut down. The bottom line shows all E. mount swap to / dev / sda5. F. check whether the mount is successful