In July last year (2019), ant group officially opened the client automation testing tool solopi, which mainly includes three modules: recording and playback (for functional testing), performance tools (for performance testing) and one machine multi control (for compatibility testing). From open source to now, we have received different voices from inside and outside the company. Some students expressed their support for the tool to improve the efficiency of the testing students. Some students thought that the automation test without the code may limit the flexibility of the test students, making this mode difficult to go far. Some students think that the tool is only a flash in the pan and simple packaging A little bit of PC tools, but the lack of practical innovation.
In fact, we started to develop this set of tools in 17 years, and now it has a history of three years. At first, solopi was just a performance testing tool, and then gradually expanded into a testing framework covering a series of mobile end test scenarios, such as functional testing, performance testing, compatibility testing, exception testing, mock testing. Along the way, solopi is just like our children Step by step growth, steady and planned progress.
In retrospect, solopi’s initial appeal was simple: testing was cumbersome, and we wanted simpler testing methods. Therefore, we start with the most troublesome performance test at that time. There are three types of performance testing tools in the past: PC driver tools, intrusive test modules, and root tools.
PC Tools:In addition to the performance testing tools provided by Android studio, most of the documents on the market are about command-line methods. Moreover, there are differences between different schemes, and many of them have errors, and there are not many tools actually formed.
Intrusive test module:Due to the need to intrude into the source code, such tools need to be packaged separately for testing, and the tool itself may also have an impact on performance.
Root tool:First of all, we need the root permission of Android system. For the Android system with more and more strict authority control, the road will be narrower and narrower.
In order to solve the problem that it is difficult to test the performance of students and the standards are not unified, we try to start with the debugging ability of Android, and research a solution that can realize the application rights promotion without root on Android phones. After a long time of research, we finally found a set of remote debugging scheme of ADB through Java（ https://github.com/cgutman/AdbLib ））This scheme communicates with Android debugging port, and obtains shell permissions through local socket, so as to achieve local application rights (hereinafter referred to as wireless ADB authorization scheme).
Combined with the wireless ADB scheme, solopi further realizes the corresponding accurate acquisition scheme for each performance index. Testing students do not need to search / develop / optimize performance testing tools any more. They only need a solopi to quickly complete the performance testing work that used to take hours in a few minutes.
After being applied in the field of performance testing, we continue to expand the application scope of solopi in wireless automation solutions. This time, we focus on the functional testing area. Traditional functional testing usually has two ways: one is to execute the test manually, the other is to write automated scripts based on test framework. The former has a huge cost, which may require a lot of manpower to cope with the accelerating product iteration; the latter requires a lot of people to test the code ability of students, which also leads to the transformation from manual testing to automatic testing, thus saving manpower. Combined with our accumulated experience in performance testing, we try to transplant the traditional automatic testing ability which can only be used for PC to mobile platform, and according to the usage habits of mobile phones, we have developed a set of simple, easy-to-use and powerful automated testing framework, that is, the current recording and playback. By recording and playing back, we can be proud to say that we can save up to 70% of the time of functional testing.
After the completion of the functional testing scheme, we have more expectations for this automated testing framework. Thus, a set of compatibility testing tools such as one machine multi control was born. As for the multi control of one machine, it originated from a chat. If the recording and playback process is split up and one machine records the use case and several or even dozens of other machines play back, the cost of compatibility testing can be greatly reduced. In the past, compatibility testing has always been a big problem in wireless testing, and the exponential improvement can be achieved through one machine with multiple controls. Through the deployment in the test room environment, we have gradually developed the habit of testing students to conduct compatibility testing through one machine with multiple controls. In the process of Alipay’s internal practice, there was a test student feedback, no one machine multi control, even do not know how to test compatibility.
In order to further reduce the test cost of the majority of practitioners, we continue to mine and optimize the entire R & D process, and have tried and made efforts on some nodes (such as data preparation, exception scenarios, small program H5, etc.). Around the core of test efficiency improvement, solopi develops and updates a series of testing tools to save labor cost.
Future trends of solopi
With the continuous development of testing technology, the trend of intelligent and simple testing is increasingly obvious. Taking MTSC conference in 19 as an example, more than half of the topics are related to AI. Focusing on the core of test efficiency improvement, we will further explore this aspect and bring AI + testing to every employee.
In our open source year, we have also found the demands of practitioners in the game field for similar tools. At present, solopi still focuses on supporting the testing ability of traditional applications. Therefore, we will further integrate the game scenarios, and we hope to be able to integrate similar unity 3D, cocos2d-x, unreal and other engines to further support, so as to better reduce the cost of game testing.
Solopi is not only a tool in the field of mobile testing, but also a product serving the majority of testing practitioners. At the same time, we have been thinking about a question: how can we further reduce the cost of learning and using test students? In the second half of this year, we plan to launch a set of more lightweight test methods, which can really let the test students test like normal use of applications, so as to achieve real intelligence and simplification.
Link to original text
This article is the original content of Alibaba cloud and can not be reproduced without permission.