Talking about single test from head to toe — on effective unit test (Part 2)


Reading guide
In “talking about single testing from head to toe — Talking about effective unit testing (Part I)”, we mainly introduce the pyramid model, why to do single testing, the stages and indicators of single testing. In the next part, we mainly introduce more wonderful contents about mock, how not to abuse mock, strategies for writing use cases, etc. Let’s take a look at it quickly~

7、 I have to say mock
test doubles
In xUnit test patterns, the author first proposed the concept of test doubles. The mock we often talk about is only one of them, and it is the most easily confused with stub. In the introduction to gomonkey in the previous section, you can notice that I don’t use mock, but stick. Yes, gomonkey is not a mock tool, but an advanced piling tool, which fits most of our use scenarios.
There are five kinds of test Doubles:
·Dummy Object
Objects that are passed to the caller but will never be used in real life. Usually, they are only used to fill the parameter list
·Test Stub
Stubs is usually used to provide encapsulated responses in tests. For example, sometimes programmed will not respond to all calls. Stubs will also record the called records. For example, an email gateway is a good example. It can be used to record all the information sent or the number of information sent. In short, stubs are generally the encapsulation of a real object
·Test Spy
Like a spy, test spy is installed inside SUT and is specially responsible for transmitting indirect outputs inside SUT to the outside. Its characteristic is that it returns the internal indirect output to the test case, which is verified by the test case. Test spy is only responsible for obtaining the internal intelligence and sending it out, not verifying the correctness of the intelligence

·Mock Object
Encapsulate the appropriate object for the set calling method and the parameters to be responded
·Fake Object
Fake objects often work together with the implementation of classes, but only to make other programs run normally. For example, an in memory database is a good example.
Stub and mock
Pile driving and mock should be the most easily confused, and we are used to using mock to describe the ability of simulated return. Habit becomes natural, so we often talk about mock.
As far as I know, stub can be understood as a subset of mock, which is more powerful:

·   Mock can verify the implementation process, whether a function is executed, and how many times it is executed
·   Mock can take effect according to conditions. For example, passing in specific parameters will make the mock effect take effect
·   Mock can specify the returned result
·   When mock specifies that any parameter returns a fixed result, it is equal to stub
However, go’s mock tool gomock only works based on the interface and is not suitable for news and Penguin projects, while gomonkey’s stub covers most usage scenarios.

8、 Don’t abuse mock
I put this part in a separate chapter to show its important significance. You need to read Xiao Peng’s “mock seven sins” on gitchat.

Two sects
From about 2004 to 2005, there were two major sects in the Jianghu: classic test driven development school and mockist (mock extremist school).

Let’s start with mockist. He advocated that all the external functions called by the tested function should be mock. That is, only pay attention to the line by line code of the tested function. As long as other functions are called, all of them are mocked and tested with false data.

In addition, the classic test driven development school advocates not to abuse mock. If you can’t mock, you can’t mock. The unit under test is not necessarily a specific function, but may be multiple functions in series. Mock when necessary.

The two sects have been fighting for many years. Their theories have their own advantages and disadvantages, and they still coexist today. Anything existing must be reasonable. For example, mockist uses too many mocks to cover the function interface, which is prone to error; Classic school, too many strings, and it is questioned that it is integration testing.

For our practical application, we don’t have to follow a certain school. We can combine them. When necessary, we can mock as few as possible without entanglement.

When is it suitable for mock

If an object has the following characteristics, it is more suitable to use mock object:
·   This object provides non deterministic results (such as the current time or current temperature)
·   Some states of objects are difficult to create or reproduce (such as network error or file read-write error)
·   Execution on object methods is too slow (such as initializing the database before the test starts)
·   The object does not exist yet or its behavior may change (such as creating a new class in Test Driven Development)
·   The object must contain some data or methods specially prepared for testing (the latter is not suitable for statically typed languages, and the popular mock framework cannot add new methods to the object. Stub is OK.)
Therefore, do not abuse mock (stub). When the other method functions are called in the test method, the first response should go into string instead of mock from the root.
9、 Use case design method
Read an article: think like a machine

This paper describes the basic idea of thinking about program design – considering input and output. When we design a case, to get the most comprehensive design, we basically consider the combination of all input and all output. Of course, on the one hand, it takes too much time and is often unenforceable; On the one hand, this is not the desired result, and the input-output ratio should be considered. At this time, it is necessary to combine theory with practice, guide practice with theory and refine theory.

First theory

  1. Starting from the previous article, when considering input and output, we must first know which belong to input and output:
    Talking about single test from head to toe -- on effective unit test (Part 2)
  2. White box & black box design
    White box method:
    ·Logical coverage (statement, branch, condition, condition combination, etc.)
    ·Path (full path, minimum linear independent path)
    ·Loop: combine 5 scenarios (skip loop, loop once, loop Max times, loop m hits, loop m misses)

Black box method:
Equivalence class: correct, wrong (legal, illegal)
Boundary method: [1, 10] = = > 0, 1, 2, 9, 10, 11 (an effective supplement to equivalence classes)
Talking about single test from head to toe -- on effective unit test (Part 2)
Talking about single test from head to toe -- on effective unit test (Part 2)
Talking about single test from head to toe -- on effective unit test (Part 2)
Talking about single test from head to toe -- on effective unit test (Part 2)

  1. Combined application
    It is difficult to implement full input and output. Instead, we think that the great gods in the industry have designed the white box and black box design method. Through careful thinking, we can judge that it is the embodiment of the methodology of full input and full output.

Therefore, I personally practice and understand the advantages and disadvantages of each of the white box and black box use case design methods. From the perspective of design coverage, condition combination > minimum linear independent path > condition > Branch > statement.

The following diagram is an early practice when I thought about use case design. Now I recall that it is over designed.
Talking about single test from head to toe -- on effective unit test (Part 2)
But in practice, we are worried about “over design”, and we can’t give the answer “what method to design to ensure foolproof”.
·Over design will also make case fragile
·In a limited period of time, we seek to maximize returns

  1. Small function & important (calculation, object processing): try to design comprehensively
  2. Heavy logic and a large number of lines of code: branch, statement coverage + loop + typical boundary processing (let’s take an example: getusergiftlist)
  3. Lead to “implementation based” and “intention based” design: the more calls inside the stub function under test, the closer it is to “implementation based” (referring to “intention based” for the second time)

10、 Intention based and Realization Based
This topic is very important.
Based on intention: think about what the function ultimately wants to do, treat the function under test as a black box, consider its output, and don’t pay attention to how it is implemented in the middle, what temporary variables are generated, how many cycles are performed, and what judgments are made, etc.
Implementation based: I also consider the input and output, and how to implement it in the middle. Mock is a good example. For example, when we write a case, we will use mock to verify whether an external method has been called in the function, how many times it has been called, and what is the execution order of the statements. The change of the program is faster than the demand. The reconstruction can happen at any time. If there is a slight change, a large number of cases fail. This is also a situation mentioned in the seven sins of mock.
What we want is based on intention, away from implementation.
Combined with practical experience, I summarize as follows:

  1. Either write well or not. Case is also code, needs maintenance and workload, so it should be written in place rather than too much. Write a pile of useless, you have to maintain, it’s better to delete.
  2. When you get a function, first ask yourself what the function is to achieve and what the final output is; Then, ask yourself where the risk of this function is and which part of the logic is not confident and is most prone to error (calculation, complex judgment, hit of an abnormal branch, etc.). These are the points we want to cover in case.
  3. Inline functions, direct get / set, few lines without logic. As long as you judge that there is no risk, you don’t need to write a case.
  4. The case to be written is determined, and then the specific use cases are designed and written in the core aspects such as branch condition combination and boundary.
    It can be understood in detail in combination with several single test case review records of news.
    Let’s look at a specific case:
  5. After getting this function, as a test classmate, I first learned the intention of the function from the developer: adding user gifts that meet the format and time
    two   Read the code, understand the code flow and several exception branches, and make a code review first
    three   Design case coverage according to the necessary exception branches
  6. The normal business process is designed according to the function intention described in the development. The case is as follows:
    Measured function
    Talking about single test from head to toe -- on effective unit test (Part 2)
    Single test case of normal path

func TestNum_CorrectRet(t *testing.T) {
giftRecord := map[string]string{
“1:1000”: “10”,
“1:2001”: “100”,
“1:999”:  “20”,
“2”:      “200”,
“a”:      “30”,
“2:1001”: “20”,
“2:999”:  “200”,
expectRet := map[int]int{
1: 110,
2: 20,
var s *
patches := gomonkey.ApplyMethod(reflect.TypeOf(s), “Getxxx”, func(_ *, _ string)(map[string]string, error) {
return giftRecord, nil
defer patches.Reset()
p := &StarData{xxx }
userStarNum, err := p.GetNum(10000)
assert.Nil(t, err)
assert.JSONEq(t, Calorie.StructToString(expectRet), Calorie.StructToString(userStarNum))
Some students will ask: but you still look at the code in the end? See how the correct logic of the code is handled, and then design the case and construct the data? And how do you know which exception branches to override without looking at the code?

A: 1. As a test student, I now write the case of development students. I really need to know which exception branches to deal with, but not limited to several in the code, but also include the exception branches I understand, which should be reflected in the case. Our case is not to prove how the code is implemented! Through single test, we can often find bugs. However, it will be developed to write single test in the future. The function designed by him must know which exception branches to cover.

  1. Well, I need to see how the normal flow of the code is, but it doesn’t mean Picking down the code to design a case. In fact, case is to design input and output by understanding the structure of input data, output format, data verification and calculation process after communication with the developer.

11、 Strategy of use case writing

We have focused on how to write a single test in order. Basically, there are three cases:
·Independent atom: mockist, overthrown by us. Of course, the bottom function may not have external dependencies, so it is enough to test it alone.
·Top down (red line): measure down from the entry function. In the process of practice, I find it difficult to execute, because I have to figure out what data and format to return for each call from the entrance. It is very difficult to string a case.
·Bottom up (yellow line): we find that the entry function often has no logic. Call another function and get the response back. So the entry function may not need to be written? We continue to look down. We look at the functions called every time, and call up the previous online and offline bugs. We find that the code part with problems is often the bottom of the call chain, especially involving calculation, complex branch loops, etc. Moreover, the function at the bottom is often testable.

Therefore, considering two aspects, we choose bottom-up design to select function writing cases:
one   The function testability at the bottom is usually good

  1. There are many core logic, especially involving calculation, splicing and branching.

12、 Solution to testability problem — refactoring
An important reason why a single test cannot be written is that the code testability is not good. If a function has eight or ninety lines, two or three hundred lines, it is basically unmeasurable, or “difficult to measure”. Because there is too much logic inside, what has happened from the first line to the last line, various function calls, external dependencies, various if / for, various exception branch handling, and the number of lines of code for writing a case may be several times that of the original function.
Therefore, it is necessary to promote the single test and reconstruct to improve testability. Moreover, through refactoring, the code structure is indirectly clear, more readable and maintainable, and it is easier to find and locate problems.
Common problems: repetitive codes, magic numbers, arrow codes, etc
The recommended theoretical books are Refactoring: improving the design of existing code, second edition and clean code
I output an article on refactoring.
We use the cycle complexity and function length of codecc (Tencent code inspection center) to evaluate the quality of code structure. We learn and practice together with development, and constantly produce results.
For arrow code, consider the following steps:
one   Use more guard statements. First judge the exception and return the exception
two   Pull judgment statements away
three   Extract the core part into a function

13、 Use case maintenance, readability, maintainability and dependability
Use case design elements
·Test internal logic separately from external requests
·Strictly validate the input and output of the service interface
·Use assertions instead of native error reporting functions
·Avoid random results
·Try to avoid asserting the results of time
·Use setup and teardown when appropriate
·Test cases should be isolated from each other and should not affect each other
·Atomicity, all tests have only two results: success and failure
·Avoid logic in the test, that is, if, switch, for, while, etc. should not be included
·Don’t protect it, try… Catch
·Each use case tests only one concern
·Using sleep less and delaying the test duration are unhealthy
·3A strategy: arrange, action, assert
Use case readability
·   The title should clearly indicate the intention, such as test + tested function name + condition + result. After a case fails, you can know which scenario fails by name without reading the code line by line. It may be others who will maintain the test code in the future. We need to make it easy for others to read
·The content of the test code should be clear. 3A principles: arrange, action and assert   It is divided into three parts. If there are many lines of code in the range of data preparation, consider pulling them out.
·The intention of the assertion is obvious. We can consider changing the magic number into a variable and naming it easily
·For a case, don’t assert too much. Be specific
·Consistent with the requirements of business code, it should be readable
Use case maintainability
·Repetition: text string repetition, structure repetition and semantic repetition
·Reject hard coding
·Intention based design. Do not refactor the business code once, resulting in a batch of case failures
·Pay attention to the bad smell of the code. See refactoring, Second Edition
Use case trustworthiness
Unit testing is small and fast. It is not to find this bug, but to try to find out whether there is a bug in each MR on the pipeline. The only reason for the failure of single test run should be the occurrence of bugs, not because of unstable external dependence and Implementation Based involvement. Long-term failure will lose the warning role of unit test. The story of “the wolf is coming” is a painful lesson.
·Non tested program defect, random failure case
·Never fail case
·Case without assert
·An unworthy case
14、 Promotion process of news unit test
We mentioned that the practice of unit testing is divided into four stages, with goals in each stage.
Phase I   Can write, all staff write, not required to write well
·The top-down promotion, from the director to the team leader, strongly supported and did not hesitate to make the team members feel high
·Quickly determine the single test frame and use it skillfully
·Combined with the development requirements, output the information under each scenario   Usage of single test framework, including assert, mock, table driven, etc
·Encapsulate http2webcontext to facilitate the generation of context objects
·Many times of training, explaining the single test theory and the use of the framework
·Each team (terminal and access layer) shall appoint a single test interface person to taste the crab first. He is the most familiar with the use of framework and writes the most cases in the early stage
·After running in the integrated use of the single test framework, the kick-off meeting was held. Some students used it on a pilot basis to ensure two consecutive iterations. These students all have case output
·Add single test related data to the summary data of each iteration: the team leader and director pay great attention to the single test data information and encourage the improvement of the number of cases and code lines

The second stage is well written, effective and written by all staff
·   The test students explored the correct use methods of mock and the correct ideas of use case design, shared them with the team, and reached an agreement after discussion
·   Pair programming, pair 2-3 developments per iteration, write cases together and improve each other.
The pairing here is flexible: for some development, it only takes half a day to tell him about the use of the framework and practice with him, so he can get started without worrying; Some developers will give the needs of the test students. After the test students finish writing the case, they will develop review learning and try to write their first case; Some development may not be acceptable at the beginning. He observed it for some time on the grounds that the requirements are not suitable for single test. He found that others have written it, which is not so difficult and beneficial to the team. He even took the initiative to find test students to teach him to write cases.
·The test students review the case submitted by the development, follow up the development and modify it, and then review it again
·For two consecutive iterations, Mr. dot and Mr. Qiao are invited to conduct a case review. The effect is very good
·For the iterative single test data analysis, pay attention to the demand coverage, personnel coverage and case increment
·The team leader continues to encourage and support single test
·The “unit test” field is added to the requirements of each iteration, which is set after evaluation by the team leader. Mr without single test will not pass, and the single test will also be reviewed

Phase III testability improvement
·Test and development study the second edition of refactoring together, and there is a sharing meeting every week
·Some backbone students give priority to refactoring their own code
·The test students are strict with each other. First, ensure that there is a single test, and then reconstruct in small steps. Each step is guaranteed by a single test
·Through the codecc scanning of the pipeline, the cycle complexity and function length must meet the standard, and manual intervention is not allowed
Phase IV TDD
·It is not guaranteed that the development students can achieve TDD first, the threshold is still very high, and they need to be proficient offline before they can be applied to business development
·Gradually promote the development and write the business code and test code synchronously, instead of adding cases after completing the business code
·Test students to practice TDD
15、 Assembly line
Talking about single test from head to toe -- on effective unit test (Part 2)
The single test should be run on the pipeline. The pipeline is equipped for the client and background to ensure that each push and Mr run once and send a report.
For the single test of go, each module of the news access layer is compiled through makefile. Because some environment variables need to be imported, I integrate go test into makefile and execute make test to run all test cases under this module.
GO = go

$(TARGET): main.go
$(GO) build -o [email protected] $^
export CFLAGS
$(GO) test $(M)  -v -gcflags=all=-l -coverpkg=./… -coverprofile=test.out ./…
rm -f $(TARGET) 
Note: the above method can only generate the coverage of the tested code file, and can not get the untested coverage. You can create an empty test file in the root directory to solve this problem and get full code coverage.
package main
import (
func TestNothing(t *testing.T) {
Pipeline plus process

cd ${WORKSPACE}   You can enter the current workspace directory

export GOPATH=${WORKSPACE}/xxx
echo “====================work space”
cd ${GOPATH}/src
for file in ls:
    if [ -d $file ]
        if [[ “$file” == “a” ]] || [[ “$file” == “b” ]]  || [[ “$file” == “c” ]] || [[ “$file” == “d” ]]
            echo $file
            echo ${GOPATH}”/src/”$file
            cp -r ${GOPATH}/src/tools/qatesting/main_test.go ${GOPATH}/src/$file”/.”
            cd ${GOPATH}/src/$file
            make test
            cd ..
  Appendix. Information
·Test Driven Development
·The art of unit testing
·Effective unit testing
·Refactoring to improve the design of existing code
·The art of modifying code
·Three practices of Test Driven Development
·《xUnit Test Patterns》
·Mock seven sins