Overview and practice of Java SDK integration testing


Project background
As a professional team that has long focused on the design and R & D of the underlying platform of the blockchain, fun chain technology attaches particular importance to product quality. Software testing can be said to be the most critical part of the quality assurance of the platform.
The underlying technology platform usually provides an SDK for external interaction. In the test phase, the system can be integrated and tested through the SDK to verify whether the functions of the system can be used normally.
Java SDK is the most widely used class in software development. Based on the practice of bitxmesh project testing, this paper introduces some experiences of Java SDK integration testing.
The main functions provided by the system are as follows:
Resource management in resource library, including file / database / model / restful interface, etc
Federated computing
Cross node resource acquisition and display
System and user management

At first, the testing tools are scattered. After adding modules or functions, they tend to create a small test project for testing. However, with more and more functions and customers, the requirements of quality assurance can not be relaxed, and building a perfect test project has become an inevitable requirement.
Taking the project as an example, this paper introduces the analysis, planning, implementation, specific module examples and the final test report of the test project in detail.

Test item analysis
In the test project, the core link is test case design. In order to design elegant test cases, we must first have a full understanding of the project itself. According to the analysis of the project function and code structure, the test project use case structure is divided into the following three categories:
User permission related test: verify that the permissions of different types of users for module operation are normal
Abnormal data test: including data exceptions and functional logic exceptions
Normal function test
Individual use cases are not difficult to implement.
However, follow-up maintenance, especially permission test and exception test, should be fully considered.
Therefore, the following two principles should be adhered to when designing test cases:
Optimize the code structure, reduce code redundancy and increase readability;
Test case data and test code are separated to facilitate separate maintenance of test cases.

Test project planning
Test framework
TestNG is an open source automated testing framework inspired by JUnit and NUnit, but it introduces some new functions to make it more powerful and more convenient to use. It draws on Java annotations to define tests to a great extent.
This project is tested with TestNG. Excel maintains the use case data, encapsulates the function to read the data, and uses the DataProvider to pass it to the test function for testing.
Overview and practice of Java SDK integration testing
Data driven testing, function parameter transfer form
Due to the strong coupling of system functions, a single use case often contains more operations. We combine the test cases of similar processes, number the location of each operation, pass parameters in the form of pair < integer, integer >, and assert the location where the exception is expected.
Exception capture and handling during testing
For permission related tests, each operation may or may not have permission. Therefore, after each operation is completed, it can be directly compared with the expectation. If an exception is caught, the error code is asserted.
For process related tests, if the previous operation has an impact on subsequent operations, assert the error code directly after detecting that the operation throws an exception, then exit the current test and continue the subsequent test; On the contrary, continue the following operation.
Automatically trigger Jenkins project build
Trigger is used to perform automatic construction. Refer to section 6 [continuous integration] for specific configuration

Project structure
Use case organization
The test cases are written in the form of tables in the internal thoughts environment, and the test cases come from the requirements documents and system test analysis documents.
Test item structure
In this project, maven, a project management tool, is used to manage builds and reports.
The project structure is as follows:

version control
The version number is consistent with the version number of the project to be tested and maintained in GIT.

Partial implementation example
There are many different types of users in bitxmesh. These users have different operation permissions for different modules and resources. Therefore, it is necessary to test the permissions one by one to see whether there are omissions or ultra vires.
In addition, some input information is limited in bitxmesh requirements, such as length / whether it is empty / repeated / special format, and boundary values and other related logic need to be tested.
Permission test (with unified error code)
Permissions in the module mainly include create / delete / view / modify / upload / download / empowerment operations.
Because creation is the basis of the following operations, use cases are specified separately to test the permission of the creation function; Other operations share a use case. When executing a use case for different users in each round, the specified user should create it first and then perform the following operations.
All permissions of a user in a module are used as a set of test data; The operation is completed normally when there is permission, and an exception is thrown when there is no permission. The status code is the specified value; After obtaining the error information, assert and compare each exception status code captured, and the current test continues.
Overview and practice of Java SDK integration testing
Abnormal data test (with special error code)
Here, the whole process of module function is tested. When the test program catches that an operation throws an exception, it returns directly without executing subsequent tests.
In order to facilitate data-driven testing, we label each operation sequence in the use case, increasing from 1; If you expect to catch an exception at the nth operation, you can pass in (n, errCode) for assertion validation

Continuous integration
Jenkins configuration
Write deployment scripts and run integration tests using shell commands
Trigger process
After git webhook detects that there is a tag or merger of trunk branches, it triggers the upstream Jenkins packaging task, generates executable files, and pushes them to nexus warehouse
After the executable file generation task is successful, the integration test task is triggered, the executable file is pulled remotely, and deployment and testing are performed
Program running environment
Docker container provided by Jenkins slave node. Because a single bitxmesh does not occupy much resources, you can run the test program and multiple bitxmesh node programs in a docker container at the same time.

Test report
In this project, Maven surefire plugin is used to generate test reports. After the project test is completed, the target / surefire reports / emailable-report.html report will be automatically generated, and the corresponding test report path can be specified in Jenkins.
Pom.xml is configured as follows:
Overview and practice of Java SDK integration testing

Permission test and exception test are two points needing attention in the test process, which need to be planned in advance in the automatic test.
The former easily leads to bloated code; The latter is difficult to consider comprehensively at one time and needs subsequent maintenance. These two problems can be solved through reasonable data-driven testing, and more perfect automatic testing can be realized in some engineering ways.
Most testers are beginners of interface function testing, and it takes some time to understand the background service testing.
The essence of Java SDK is to encapsulate the system call interface into a class library for the client to use; When conducting integration testing, you need to be familiar with the SDK usage documents in advance, and then plan through the hierarchical architecture.

Introduction to the author
Dai Jiaxiong
From the data grid lab team
Mainly responsible for the overall quality assurance of distributed data sharing platform