Baidu app fluency whole process quality monitoring practice (1) fluency status analysis

Time:2021-7-19

preface

Fluency testing is a deep field in client performance testing technology, so Baidu app brings you a series of articles on fluency whole process quality monitoring practice.

It includes:

Series (1) analysis of fluency,

Series (2) fluency index selection,

Series (3) fluency online and offline monitoring practice,

Series (4) fluency competitive product evaluation scheme.

Hope to help you in the fluency performance testing direction of learning and practice.

background

Why focus on fluency? App is easy to crash and web news can’t be opened, which are pain points. Users will complain or even unload. After continuous optimization, improving stability and reducing the white screen rate, when these pain points are not so painful, we need to focus on the “cool points” of users. One of the typical problems affecting the cool points is the carton problem.

Through user feedback analysis, the test students found that Caton had an impact on the user experience in different business scenarios, and in some cases, the user’s mood was poor. However, in fact, when case by case processing, it was often due to the failure of offline reproduction, insufficient clues provided by users and other reasons, so it was necessary to establish separate monitoring indicators for recall.

On the other hand, with the continuous emergence of high-performance mobile phones, in order to improve the user interaction experience, the design is no longer a simple pop-up box or loaded interaction, dynamic effect and other “dazzling” interaction scenes are more used. While bringing advanced experience, will it create a burden on low-end computers? Will not give people not only “dazzle”, but the card can not use the feeling? Therefore, we need a set of good fluency monitoring standards to master the balance between advanced interaction and Caton

Research on industry fluency monitoring scheme (Android)

1. Introduction of basic fluency concept

1.1 ideal frame rate:

60fps, limited by display refresh rate 60Hz

1.2 ideal frame length:

1/60≈16.6ms

1.3 Vsync mechanism

Vsync can be simply regarded as a timing interrupt. The system will send a Vsync pulse signal every time it needs to draw, and CPU / GPU will process the drawing immediately after receiving the signal.

2. Industry scheme research

Baidu app fluency whole process quality monitoring practice (1) fluency status analysis
Baidu app fluency whole process quality monitoring practice (1) fluency status analysis

3. The principle of monitoring

3.1 statistical frame length based on Vsync: statistical frame length, SM, SF:

The choreographer class is to receive the system vertical synchronous signal (Vsync signal). Each time it receives the Vsync signal, it sequentially performs three operations of view, such as input, animation, and draw. Then it waits for the next signal and sequentially performs three operations again. If the draw operation is not completed on time when the second signal arrives, the interface will not be updated and the contents of the first frame will be displayed. This means that the frame is lost, which is the cause of the picture stuck. Therefore, we can add our own callback to the choreographer class. Through the doframe function of this callback, we can count the number of frame drawing times in one second (smooth value SM), drawing time (time between two doframes, frame length) and frame loss SF.

3.2 based on looper:

The frame length is obtained by matching logs printed by looper of UI thread. Similar to the Vsync scheme, only when the UI thread is seriously blocked, data loss may occur( The impact on UI thread is also a point to be balanced.)

three point three   Stack monitoring:

The single thread grabs the stack periodically, and uploads the latest stack when the frame length exceeds the specified threshold based on Vsync or looper mechanism. However, due to the single thread real-time stack grabbing, the performance of the application itself will be degraded, which is not suitable for long-term large-scale online use.

three point four   Precautions for monitoring (actual measurement experience)

In the actual test, it is found that when the app is still, especially when the web page is still, the SM value may become lower. For example, when it is close to 30, the SF value and frame length may exceed the ideal value. The reason is that although the user does not operate the interface, there may also be downloading in the background, animation outside the screen display area and other behaviors. The overall interface display does not show a stuck, but it may affect the loading that the user cannot perceive with his naked eye.

——In different stages and different scenes, the absolute value of the same fluency index has different response accuracy to the user’s actual experience, so the accuracy of the response is differentIt is recommended to distinguish scenarios and phases for monitoring.

reference material

  1. Xiao Zhu: Android App performance evaluation analysis – Fluency
  2. Htkeepmoving: mobile app performance evaluation fluency evaluation
  3. Tencent: [Tencent TMQ] gt3.1 simplify your app performance test
  4. Wechat reading: Caton monitoring system
  5. Tencent perfdog: perfdog performance dog help document
  6. Whbsspu: why is the page smooth when the frame rate reaches 60fps?
  7. Egos: introduction of Vsync mechanism in Android
  8. Markzhao’s home: blockcanary – easy to find the culprit of Android App interface jam (Android performance monitor)

The author of this paper:
MQA-sherryshare


Enter “Baidu app technology” in wechat – Search page to pay attention to wechat official account; Or use wechat to identify the following QR code, you can also pay attention to it.
Baidu app fluency whole process quality monitoring practice (1) fluency status analysis