[original] KVM QEMU analysis of Linux Virtualization (1)



  • Read the fucking source code!–By Lu Xun
  • A picture is worth a thousand words.–By Golgi


  1. KVM version: 5.9.1
  2. QEMU version: 5.0.0
  3. Tools: source insight 3.5, Visio

1. Overview

  • Starting from this article, we will start a series of research on virtualization, which will probably involve the analysis of arm64 virtualization support, KVM, QEMU, etc;
  • Virtualization related practices and operations include and only have: the use of virtual machines such as VMware / VirtualBox, the use of QEMU, and the modification of QEMU source code to simulate IO devices;
  • Show me the codeAll from the source code;

This article as the beginning, from the macroscopic aspect to carry on the introduction, has a preliminary understanding, does not involve the concrete principle analysis.

2. Concept

2.1 virtualization

What is virtualization?

  • Virtualization is a resource management technology. In a non virtualized system, a single operating system manages and uses all hardware resources. In a virtualized system, hardware resources can be abstracted and divided into multiple virtual entities to support multiple operating systems. Multiple operating systems can share all physical hardware resources to maximize the utilization of physical resources;
  • Virtual Machine Motior(VMM)Virtual machine monitor, also known asHypervisorTo manage physical resources downward and provide logical resources for different virtual machines upward;
  • Virtual Machine(VM)The virtual machine can run different OS (guest OS) according to its own choice, and it will think that it has exclusive hardware;
  • The advantage of virtualization is that it can improve the utilization rate of resources. For example, the current computer configuration resources are very high, and the actual utilization rate is relatively low. If we carry out unified management and virtualization, we can support more users to make rational use of them;

2.2 software virtualization and hardware virtualization

2.2.1 software virtualization

Through software simulation to achieve the VMM layer, such asQEMUHere is an example:

  • Take a typical scenario (arm + Linux simulation environment): use QEMU to simulate arm64 processor in PC Ubuntu system, and run guest OS in arm64, assuming that guest OS is also Linux;
  • Without the support of hardware virtualization, QEMU essentially completes the work of binary translation. How to understand this problem? For example, when guest OS is running, both app and OS think that they are running in arm64, and the execution files are generated by cross compiler. We all know that different processor architectures and instruction sets are different. Programs running on arm cannot be executed when they are placed in x86. The emergence of QEMU can solve this problem and translate them mechanically;
  • QEMU’s translation process is: toGuest code instructionTranslated intoTCG (tiny code generator) intermediate codeAnd finally translated intoCode instructions supported by host architecture

2.2.2 hardware virtualization

It is obviously inefficient to translate instructions by pure software behavior. The support of hardware virtualization can improve the overall performance. Hardware virtualization means that the processor itself provides the ability to let the client instructions run independently.

  • KVM (Kernel-Based Virtual Machine)The virtual machine based on kernel realizes the virtualization of CPU and memory, and the interception of hardware I / O virtualization, and the I / O of guest is blockedKVMAfter interception, it will be handed over to QEMU for processing;
  • KVMIt is a module of the kernel, which can turn Linux into aHypervisor
  • KVMThe host processor itself needs to support virtualization extensions, such asintel VTAMD-VEtc;

2.3 semi virtualization and full virtualization

  • Semi virtualization(Para-Virtualization): the client operating system knows that it is running in a virtual environment, and makes customized modifications to work with hypervisor. The advantage is that the semi virtualized architecture is more concise, and the performance has certain advantages. The disadvantage is that the client OS needs to be modified, and the user experience is biased;
  • Typical semi virtualization technologyvirtioThe host / hypervisor and the client are required to install corresponding drivers;
  • Full virtualization(Full Virtualization): the client operating system does not need any changes, and it is easy to use. Because full virtualization needs to simulate a complete platform, which is the same as the physical platform, it also increases the difficulty of hypervisor design;

2.4 type 1 virtualization and type 2 virtualization

  • From the perspective of software framework, virtualization can be divided into type 1 and type 2 according to whether the hypervisor is directly on the hardware or on the host operating system;
  • Type1 Virtualization:native/bare-mental Hypervisor, directly control hardware resources and clients, typicallyXen
  • Type 2 Virtualization: hypervisor runs on the host operating systemVMware Workstation, KVMHypervisor is just an application of host operating system;

2. KVM QEMU framework

From the above classification of virtualization, our research objectives are as followsKVM+QemuIs a full virtualization solution (type 2) using hardware virtualization technology.

  • Qemu (Quick Emulator): is the user mode component of virtualization solution, which has two modes: 1)Emulator, simulator, simulating all kinds of hardware, using binary translation technology; 2)Virtualiser, virtual machine, viaioctlAndKVM kernel moduleInteractive, complete the virtualization function;
  • QemuFor eachVM virtual machineCreate a process for eachVcpu virtual CPUCreate a thread, and the guest’s system and application run in thevCPUabove;
  • QemuIt can simulate the I / O function, and this part of the functionKVMIt may not be fully supported. The execution process is as follows: virtual machineVMThe program in performs I / O operations,VMExit and enter KVM. KVM will judge and give the control right to KVMQemu, byQemuTo simulate the I / O device to respond to the I / O request of the program;
  • KVM kernel module, which relies on the virtualization support of the underlying hardware, is mainly used to initialize the CPU hardware, open the virtualization mode, run the virtualization client in the virtual machine mode, and provide some support for the operation of the virtualization client;
  • KVM kernel module realizes the virtualization of CPU and memory, while KVM module is usually not responsible for the virtualization of peripheral io. Only KVM kernel module is responsible for the virtual devices with high performance requirements, so there is a combination scheme of KVM + QEMU;

This article only when the literacy paste, as for the in-depth analysis of specific technical details, the follow-up will be updated from time to time.

3. Reference

"KVM practice - principles, advances and performance tuning"

Welcome to pay attention to the official account number and update the kernel mechanism regularly.