On the display process of mobile view

Time:2019-12-2

Author: seven Android Development Engineers

With the development of science and technology, all kinds of mobile terminals have become an indispensable part of people’s daily life. People use mobile products to work, socialize and entertain The fluency of mobile interface has become one of the important factors that affect user experience. Have you ever thought about how to realize the smooth picture displayed by the mobile terminal?

In this paper, through a brief analysis of the mobile view display process, we can help developers understand the logic of view rendering and better optimize their own app.
On the display process of mobile view

The image above shows a complete page rendering process. Through the above figure, we can preliminarily understand how the logic behind each frame of page is executed step by step from the compilation of code layout to the presentation to the user.

How the screen looks

Pixel points

In fact, the pictures displayed in the electronic screen are all composed of “small dots”, which are called “pixel dots”. Each pixel has its own color, and each complete picture is formed by connecting and splicing them.

Each pixel generally has three subpixels: red, green and blue. According to these three primary colors, we can modulate a variety of colors.

On the display process of mobile view

Large TV set

Unlike today’s flat-panel TVs, black-and-white TVs or large rear projection color TVs used to carry large “backs”. “Big back” TV is actually cathode-ray tube TV, commonly known as picture tube TV. The imaging principle is that the electron beam (cathode ray) emitted by the electron gun passes through the focusing system and the deflection system to the designated position on the screen coated with the fluorescent layer. At every position bombarded by the electron beam, the fluorescent layer will produce a small bright spot, and finally the small bright spots will form an image and display on the TV screen.

On the display process of mobile view

On the display process of mobile view

This is also the reason why the screens of big TV sets used to be arc-shaped. Because the closer to the circle, the closer the distance from the side length to the center, the more uniform the image. So why does it cause problems when the magnet is close to the TV? That’s because the magnet will interfere with the normal trajectory of the electron beam, and when it is close to the screen, it may also make the fluorescent layer of the screen magnetized, showing abnormal spots.

The picture below shows the drawing process of electron beam after the camera is slowly played.
On the display process of mobile view

On the display process of mobile view

LCD and OLED

With the continuous progress of science and technology, the volume of TV, mobile phone and computer is becoming thinner and thinner, and the way of ray tube imaging is gradually eliminated. At present, LCD and OLED are two kinds of screens that occupy the mainstream position in the mobile phone market.

On the display process of mobile view

The full name of LCD is liquid crystal display. OLED is called organic light emitting diode. There are significant differences between the two:

1. Different imaging principles
LCD is based on the white backlight through the color film, while OLED is based on the self illumination of each pixel.

2. In terms of power consumption
The power consumption of LCD is high. Even if only one bright spot is displayed, the backlight of LCD needs to be always on, and it is easy to leak light. And each pixel of OLED can work independently, and can emit light by itself, so the device using OLED can be made thinner, or even bent.

3. In terms of production
LCD uses inorganic materials, OLED needs to use organic materials, so OLED has higher production cost and shorter service life than LCD.

Graphic display core GPU

Compared with CPU, GPU has more computing units and is better at large-scale concurrent computing, such as password cracking, image processing, etc. CPU is to follow the order of von Neumann architecture storage program execution, in the large-scale parallel computing capacity, is more limited, so it is better at logic control.

On the display process of mobile view

Application programming interface API (OpenGL)

Before there is no unified API, developers need to write a variety of custom interfaces and drivers on a variety of graphic hardware, which is a huge workload.

In 1990, SGI (Silicon Valley graphics company) became the leader in the field of workstation 3D graphics, and transformed its API into an open standard, namely OpenGL. Later, SGI also contributed to the creation of the OpenGL architecture review board (OpenGL ARB).

On the display process of mobile view

Vertical synchronization

When we find that the page is stuck during the use of mobile app, it is likely that the page is not updated within 16ms. In fact, the cooperation between human eyes and brain can’t perceive the picture update of more than 60 FPS. 60fps is equivalent to 60 frames per second, so each page needs to be updated to other pages within 1000 / 60 = 16ms, so that we won’t feel the page jam.

Without Vsync, the following situations may occur:

On the display process of mobile view

As shown in the above figure, in the absence of Vsync, when the second frame needs to be displayed, it has not been processed, so the first frame is still displayed in the display. This will cause the frame to be displayed for more than 16 ms, resulting in page jam.

In order to keep the frame generation speed of CPU and GPU consistent with that of display, Android system will send Vsync signal every 16ms to trigger UI rendering update.

On the display process of mobile view

From the above figure, we can see that every 16ms, Android will send a Vsync signal. After receiving the signal, the CPU starts to process the content of the next frame. After the CPU processing, the GPU will be rasterized. At this time, the screen will display the processed page of the previous frame. So repeatedly, you can display a specific picture in the page. The premise to ensure smooth picture is that CPU and GPU can not spend more than 16 ms processing a frame, otherwise the following situations will occur:

On the display process of mobile view

When the CPU and GPU process a frame for more than 16 ms, in the first display, because the GPU processes B picture for a long time, resulting in the system sending Vsync signal, the display can not display B picture in time, but repeatedly display a page, resulting in a jam.

In addition, in the second display, because the a buffer is still used by display, it cannot start processing the next frame of the page after receiving the Vsync signal, resulting in the idle CPU in this period. In order to avoid this kind of time waste, three caching mechanisms appear:

On the display process of mobile view

As shown in the above figure, in the three cache mechanism, when a cache is used by display and B cache is processed by GPU, the system will send Vsync signal and add a new cache C to cache the content of the next frame. Although this method can not completely avoid the repeated display of page a, it can make the display of subsequent pages more smooth.

Drawing process of view

The drawing of view starts from the performtraversals() method of viewrootimpl. The overall process is roughly divided into three steps, as shown in the following figure:
On the display process of mobile view

measure

Control measurement starts with the performmeasure () method. In this method, childwidthmeasurespec and childhighmeasurespec are used to determine the width and height, respectively.

Measurespec is an int value, which stores two information: the lower 30 bits is the view’s specsize, and the higher 2 bits is the view’s specmode.

There are three types of specmode:

1.UNSPECIFIED

The parent view has no restrictions on the child view. You can set the view to any size according to the wishes of the developer, which will not be used in the general development process.

2.EXACTLY

The parent view specifies an exact size for the child view, which is determined by the value of specsize.

3.AT_MOST

The parent view specifies a maximum size for the child view, which is specsize.

By observing the measure () method of view, it can be found that the method is modified by final, so the subclass of view can only complete its measurement logic by overloading the onmeasure () method.

On the display process of mobile view
In the onmeasure() method:

On the display process of mobile view

Call the getdefaultsize() method to get the size of the view:
On the display process of mobile view

The second parameter measurespec in this method is passed from the measure () method: get the corresponding value through getmode() and getsize() parsing, and then assign a value to the final size according to specmode.

However, the above is only a measurement process of a simple control. In the real measurement process, because a page often contains multiple subviews, it needs to cycle through the measurement. In the ViewGroup, there is a measurechildren() method, which is used to measure the subviews:
On the display process of mobile view

The method call chain of the overall process of measure is as follows:

On the display process of mobile view

layout

After the measurement of performtraversals() method, enter the layout process:

performLayout(lp,desiredWindowWidth,desiredWindowHeight);

The main function of this process is to put the corresponding view in the appropriate position according to the size of the sub view and the layout parameters.

host.layout(0,0,host.getMeasuredWidth(),host.getMeasuredHeight());

As above, the layout () method takes four parameters, which are clockwise, top left and bottom right. This coordinate is for the parent view, with the top left as the starting point, and the width and height measured before are passed in. After that, let’s go to the layout () method and observe:
On the display process of mobile view

We use the setframe () method to assign values to the four variables to determine whether the position of the view changes and whether it needs to be replayed. In addition, the onlayout () method is also called.

After entering this method, we can find that it is empty. This is because the specific position of the child view is relative to the parent view, so the onlayout of the view is empty.
On the display process of mobile view

After entering the ViewGroup class, we can see that this is actually an abstract method. In this case, the subclass of ViewGroup needs to override this method:
On the display process of mobile view

draw

The drawing process is mainly as shown in the following figure, which is also a process of traversing sub view drawing:

On the display process of mobile view

It should be noted that the ondraw() method of view is empty, because the content of each view is different, and this part is handled by subclasses according to their own needs, which is more reasonable:

On the display process of mobile view

The whole process of Android rendering mechanism

On the display process of mobile view

1. App builds commands and data needed for opengl rendering in UI thread;

2. CPU uploads (shares or copies) data to GPU. (generally, there is display memory on PC, but the memory of ARM embedded device is GPU and CPU shared memory);

3. Notify GPU to render. Generally speaking, the real machine will not block waiting for the GPU rendering to finish, and will return to perform other tasks after the notification is finished;

4. Inform surfaceflinger layer composition;

5. Surfaceflinger starts to synthesize layers.

summary

The technology of mobile terminal is developing rapidly, and the optimization of screen display is a continuously developing practical subject, which runs through every developer’s daily work. In the future, the personal push technology team will continue to focus on the performance optimization of the mobile terminal, and share relevant technical dry goods for you.