The game project adopts agile development, version development iteration is very fast, basic 1-2 weekly version
Necessity of performance test
Number of performance issues in phases throughout the project
Performance problems do not occur at the beginning or suddenly appear one day, but accumulate with our development progress;
Later, we hope to solve problems for months or even years in a few days, but in fact, the results are not satisfactory. And the same problem, the same person, at different times to deal with the experience and time is completely different.
Therefore, the performance problem seems to be a technical problem of the R & D team, but in essence it is the development process of the R & D team
If we can standardize the process so that each version has a data display, and once problems are found and handled in a timely manner, the optimization time in the future can be greatly reduced. However, the performance of each version of human resources is relatively weak, so it can be handled in an automatic way. Will the automatic operation affect the performance data we get, Now let’s explore;
Impact of automation on application performance data
Comparison of the first group of tests
1. Open perfdog to record the performance data of manual running function and automatic running function
2. The automation function used this time is airtest
1. The airtest ide connection is not opened, and the function of manual running is enabled
2. Open airtest ide connection and run manually
3. Open airtest ide connection and use the automatic script running function
4. Disconnect airtest ide
5. Close airtest ide process
Will only run a small combat function, very short time
The disconnection of the following test cases refers to:
Let’s take a look at FPS first
Obviously, we find that whether or not to run the game functions automatically has little effect on FPS
Let’s look at the memory
It is found that automation has no effect on memory, and it is almost the same for memory if automation cannot be opened
Let’s look at the CPU
We found that the total CPU utilization rate increased significantly when the IDE connection of airtest was opened, and the utilization rate of total CPU was also increasing when running automation scripts. However, the CPU utilization rate of APP has almost no impact.
This is because when the airtest IDE is connected, the IDE will use minicap service to obtain the screen shot of the phone, which will affect the overall utilization rate of CPU. When running the script, airtest needs to search and match images, so it also takes up CPU. However, the usage rate of app will not be affected.
Comparison of the second group of tests
This test does not apply to automated scripts, the impact of IDE is compared separately
1. The static page is not connected to the airtest ide
2. Connect the static page to airtest ide
3. The static page disconnects airtest IDE and does not exit ide
4. Disconnect airtest ide from static page and exit ide
Whether IDE is turned on or not has no effect on the FPS of the application
Memory doesn’t matter
Just like the conclusion of the first group, it is also true that turning on the IDE will affect the total CPU utilization rate. It should be noted that the performance consumption will still remain after the IDE is disconnected from the mobile phone, because the mincap service has not been interrupted in practice. It is necessary to exit and shut down the IDE CPU before it can return to normal.
The third set of data
The selected one is a mobile app, not a game
We found the same conclusion as above
Standardized CPU utilization recommended
Why is this value recommended as a measure of CPU utilization? Because it is found that normalization is more suitable for automation and is more accurate. Documents on normalized utilization rate:
Introduction to standardized utilization rate
We can use the automatic way to obtain the performance data of the application. This is because the data we get is for a single application, so the automatic operation will not be within the application. However, the access to the automatic SDK should be considered separately, and the resources consumed by the SDK will be counted on the application head.