Introduction: quality automation of idle fish trading
As a vertical trading community app, Xianyu has complex and diverse business scenarios: complex and changeable transaction modes involving C2C, recycling consignment, rental, meeting transaction, inspection guarantee, etc. For example, the inspection process:
- Involving 39 state machine nodes
- Across 10 + applications
- Cooperation involving 6 business units
- Dozens of interfaces are involved
We need to ensure that each interface and each scenario are feasible. If there is a little problem, it will involve the taste of RMB. In practical work, we encounter various problems, and the thorny problems are as follows:
Under the fast iteration mode of business first win, the test and verification are all carried out by manual main force. After testing new functions, you have to return to the old functions. It takes several man days for a small demand, and the PTM version has to return several times. The ROI is not optimistic. The following two problems are prominent:
- The trading business is strongly dependent on the middle office, with high communication cost, difficult cross team cooperation and low iterative efficiency. How to be self consistent in the test environment?
- How to support the steady iterative online and daily regression verification of requirements under complex and diverse transaction modes?
Test strategy – Automation
The quality infrastructure of idle fish is speeding up. For the various trading modes of idle fish, it is not feasible to rely on manpower. If you are tired, changes and missing risk assessment also occur from time to time. In this regard, we explored and compared several different schemes according to the strategy of interface – > link to ensure the whole link on the basis of ensuring that each interface is OK.
For each large-scale application, the number of interfaces will increase, the frequency of code changes will increase, and the system will be reconstructed irregularly. How can the quality of this interface be guaranteed? The traditional way of writing scripts costs too much manpower and time. In the actual testing process, we have explored some new ideas of interface testing. At present, the recognized effective way in the industry is the automatic test based on drainage playback. The implementation scheme has different opinions in the industry, and each has its own words, but everything changes. It is simple and clear to quote the following summary
One is the idea of black box test, which collects the online traffic (mainly the request parameters and results) when the online interface requests, then triggers the request again with the collected traffic in the same environment as the online environment (database sharing, etc.), and then asserts whether the requested return value is consistent with that during recording. This method is more suitable for testing get type interfaces, and write requests are easy to cause data pollution. In addition, the data status of collected traffic (data timeliness), environmental dependence (RPC calls requested by various middleware and interfaces) and other factors, so this test method has some limitations, It can not meet the complex requirements in the actual test scenario.
Another idea is relatively white box. It mainly collects the return results of external middleware or RPC calls that the code depends on during operation through intelligent mock means. When the traffic is replayed, it can mock the contents that may change in the external dependence of the native program, so as to make the test pay more attention to the code logic of the local interface.
Within Alibaba group, based on the idea of traffic playback, two different traffic recording and playback schemes are mainly implemented, one is doom / Blizzard based and the other is phoenix based on JVM sandbox. Both implementations rely on JVM AOP.
- Apocalypse / Blizzard
For Apocalypse / blizzard, doom is used for traffic recording at the bottom. Its principle is as follows
The main processes are:
- Collect the input parameters, return values and sub calls of the main call (the entry method during collection or playback) and the input parameters and return values of the sub call (a method call during application execution, and the collection machine will collect the input parameters and return values of the method for mock during playback) in the AOP mode of ASM through the client hung in the JVM by the Java agent, Then upload the collected data to the server (offline mode);
- During playback, client will execute local logic of the interface after receiving playback request of the interface. For sub call, it will mock with the collected input participation result;
- Compare the collected traffic with the playback result data.
In the doom mode, the business application system needs to introduce the jar package, modify the startup class, and modify the JVM to mount the agent, which is partially intrusive.
-Phoenix Phoenix is also a traffic recording scheme implemented by JVM AOP. The concept is similar to doom. The bottom layer of Phoenix's overall architecture is based on JVM sandbox (a non-invasive runtime AOP solution for JVM platform open source by Ali, which enhances the AOP function at the method level through bytecode). During recording, the calling method, input parameter, return value and calling sequence are recorded, which are stored in a chain data structure. During playback, the interface logic execution and sub call mock are carried out< br />![ Phoenix record playback.png]（ https://gw.alicdn.com/imgextra/i2/O1CN01m49rqS1rsh7EMIakW_ !! 60000000005687-2-tps-442-331. PNG) < br / > Phoenix recording and playback < br / > Phoenix does not need code intrusion and modification, and does not need to modify application startup parameters. Relatively speaking, it has little impact on business code, but it has application structure requirements. Considering the cost and risk, as well as our application structure, idle fish is guaranteed by Phoenix traffic recording and playback based on sandbox, and the online process card point is changed< Br / > in the process of R & D, we will also encounter various traffic playback problems, such as expired use cases, which need to be clearly re recorded manually. We are now using a scheduled task to automatically clear and re record< Br / > here is an example of our scenario: < br / >[ image.png]( https://gw.alicdn.com/imgextra/i3/O1CN01bR7Yqe1qfaA29uZCx_ !! 6000000005523-2-tps-1318-418.png)<br /><br />
In the process of interface testing based on traffic recording and playback comparison, we found that this mechanism is more practical for the quality assurance of a single application, but there are some deficiencies in cross application link verification, core write operations, external calls, system reconfiguration and scheme transformation, and link level solutions follow.
- Thub + microservices
In the test environment, for the strong dependence on the upstream and downstream of the whole link, one of the measures is to develop the test service capability and establish the self consistent capability. In the test environment, the dependence on the outside such as the trading center and rookie wrapping can be decoupled, and the test environment can carry out the whole link closed loop.
The primary task of landing is to sort out the business full link nodes:
- Each MTop interface on the backbone link and the upstream and downstream dependencies of the interface
- Dependence of internal applications, medium-sized applications and external merchants
- Data flow and tddl combing
The business is sorted out completely, and the test service interface is developed. Here are some link cases we intercepted:
At the same time, for example, in the test environment, due to the unstable test environment of the relying party, block testing, we provide a test service interface for packaging, expose the service capabilities such as ordering and inspection, and build it into the idle fish quality platform for development, testing and use in the R & D process.
- Tiansuan platform
The sky computing platform uses the shadow database and the full link voltage test mode to isolate the online business data from the test data, and the test library copies part of the data of the online library. The main implementation method is to solidify and simulate the online scene, execute the whole link, and compare all data changes in the process of execution. The user can select the baseline and changed version of any code version for comparison.
The day computing capability can basically meet the transaction link of idle fish. Idle fish has established a shadow database related to the main link. The shadow link is being debugged for full link patrol inspection at the transaction server. At the same time, the shadow link has problems such as the expiration of shadow data caused by service changes. This scheme is mainly used for services with relatively stable services. New services or services that are constantly iterative and updated do not have this scheme.
To sum up, in the current idle fish transaction, the interface layer uses the traffic recording scheme based on JVM sandbox, the shadow link is used for daily patrol inspection, and the service-oriented ability of business arrangement for R & D process self-test and link automation is used.
On the basis of improving the infrastructure, we will continue to explore the testing solutions in the direction of full-end intelligence of shuttle and service end, hoping to release more technical sophomores from repeated work, ensure free fish trading from the three-tier quality network of governance, prevention and control, and let users sell and buy free fish at ease. Look forward to communicating with you the different test schemes in the industry! At the same time, I would like to thank the ability support provided by the teams such as doom, sandbox, Phoenix, apocalypse, Blizzard, full link voltage test and Thub!
This article is the original content of Alibaba cloud and cannot be reproduced without permission.