Introduction to Android MMAP file mapping to memory

Time:2020-2-28

Link to this article: introduction to Android MMAP file mapping to memory

In Android development, we may need to record some files. For example, log file. If stream is used to write files, frequent operation of file IO may cause performance problems.
In order to reduce the frequency of writing files, we may cache a certain number of logs and write them to the files at one time. If the app exits abnormally, we may lose the log information in memory.
So what is a more secure way to write files, which can not only reduce IO, but also ensure that data is written to files as much as possible?

MMAP introduction

MMAP concept

MMAP is a method of mapping files in memory, that is, mapping a file or other objects to the address space of a process to realize the one-to-one mapping between the disk address of a file and a virtual address space of a process.

Features: after such a mapping relationship is realized, the process can read and write this section of memory in the way of pointer, and the system will automatically write back the dirty page to the corresponding file disk, that is, the operation of the file is completed without calling read, write and other system call functions. On the contrary, the modification of kernel space to this area also directly reflects user space, so that file sharing between different processes can be realized. As shown in the figure below:

mmap简介

MMAP memory mapping principle

Generally speaking, the implementation of MMAP memory mapping can be divided into three stages:

In the virtual address space of a process, a free and continuous virtual address is found as the mapping area;
Call the system function MMAP to map the physical address of the file to the virtual address of the process;
The application process accesses the mapping area, causes page missing exception, and copies the file content to the physical memory (main memory).

Advantages and disadvantages of MMAP

Only one data copy: in case of page missing exception, directly copy the data from disk to the user space of the process, skipping the page cache.
The efficient interaction between user space and kernel space is realized: the modification operations of the two spaces can be directly reflected in the mapped area, so that they can be captured by the other space in time.
It provides a way for processes to share memory and communicate with each other.

Whether it is a parent-child process or an unrelated process, it can map its user space to the same file or to the same area anonymously. Thus, the purpose of inter process communication and inter process sharing can be achieved by changing the mapping area.

At the same time, if both process a and process B map area C, when a first reads C, it copies the file page from disk to memory through page missing; however, when B rereads the same page of C, although page missing exception will occur, it is no longer necessary to copy the file from disk, but can directly use the file data saved in memory.

MMAP notes

For large files, memory mapping is faster than ordinary IO streams, while small files are not;
Don’t call mappedbytebuffer. Force() method often. This method forces the operating system to write the contents of memory to the hard disk. Therefore, if you call force() method every time you write the memory mapping file, you can’t really benefit from the memory mapping file, but it is similar to disk IO.
The operating system is responsible for reading and writing memory mapping files. Therefore, even if your Java program hangs up after writing to memory, as long as the operating system works normally, the data will be written to disk.
If the power supply fails or the host fails, it is possible that the memory mapping file has not been written to the disk, which means that some key data may be lost.

Reference resources

  • https://stackoverflow.com/questions/258091/when-should-i-use-mmap-for-file-access
  • https://www.jianshu.com/p/187eada7b900
  • https://juejin.im/post/5c3ec9ebf265da61223a93de#heading-0
  • https://stackoverflow.com/questions/30180268/android-ndk-mmap-call-broken-on-32-bit-devices-after-upgrading-to-lollipop
  • https://stackoverflow.com/questions/33897711/android-mmap-fails-with-out-of-memory

Binder in Android also uses MMAP. When binder delivers data, it only needs to copy it once to deliver it to another process. Refer to the introduction of binder mechanism

Using MMAP in Android

MMAP can be used in Android through RandomAccessFile and mappedbytebuffer. Refer to drone development log logging tool

adoptrandomAccessFile.getChannel().mapGet toMappedByteBuffer。 Then call the put method of ByteBuffer to add data.

Recommended Today

Swift advanced 08: closure & capture principle

closure closurecanCapture and storageOf any constants and variables defined in their contextquote, this is the so-calledClose and wrap those constants and variablesTherefore, it is called“closure”Swift can handle everything for youCaptured memory managementOperation of. Three forms of closure [global function is a special closure]: a global function is a closure that has a name but does […]