Testops: thinking about selenium UI automation — why is UI automation difficult?

Time:2022-5-6

In fast iterative products or teams, UI automation is usually a seemingly beautiful and practical tool (even most of them are not even chicken ribs).

The reasons are as follows:

1、 Limited effect

Usually, teams who just heard about UI automation and want to engage in UI automation think that [UI automation] is equal to [reducing labor and improving efficiency], which is no big mistake, but usually they also think that the reduced labor cost and improved efficiency will be very high, so they will have very high expectations for UI automation, which is very problematic. After all, the reality is very cruel, and the real effect of UI automation is not so good.

This effect is not so good, which is usually reflected in two aspects:

1. Itself can not fully meet the complex business code framework. In addition to the deficiencies of common elements such as ID and name, some frameworks are complex and require very high requirements for development in order to add the required information. For example, Vue, a very easy-to-use framework, is well encapsulated, which also means that it is very difficult to modify Vue. Moreover, some front-end code is directly generated dynamically, which makes the already sad automation even worse.

2. Some students will say that if it is not applicable, you have to use it forcibly. If it is not easy to use, you have to blame tools. It’s really yours. Indeed, the first point is caused by the incompatibility between the tested system framework and selenium, but the second point is the insurmountable problem of automation at present. In general, UI automation can only reveal the known problems, and basically can’t check out new bugs. This feature is innate. When we do our own tests, we also know that many bugs are actually found outside the test cases. After all, UI automation is a piece of coding. It can not report bugs beyond the preset, but can only reveal the known problems.

In fact, if you can do well, it is already very good. It can still provide very good help for the company and the team. However, in addition to the above reasons at the framework level, the reasons at the company level are also very key.

2、 Insufficient attention

In many companies that have set up UI automation posts, usually there are only one or two students who spend less than half of their time writing automation scripts, and these students have high hopes that they can make a UI automation system with wide coverage, high test passing rate, stable continuous integration, and the leaders take it out and force them to install it.

It’s cold to think about it, but this is the real situation. I want horses to run and horses don’t eat grass.

In addition, most companies that recruit UI automation give low salaries, generally between 8-20 (more than 15 are generally in large factories). Let’s think about it in another way. If the monthly salary of a company’s main project is 20K, what do you think of the company’s R & D level? The answer is self-evident. Many students are not satisfied. Your UI automation test is compared with serious development. It’s really yours. I really want to say that if UI automation wants to be implemented, the recruitment standard should be based on the development level, otherwise it will waste people and money, which is meaningless.

Of course, companies that offer high wages and do UI automation full-time are not without them, but the effect is hard to say, because it involves the content of the executive level.

3、 Painful maintenance

Many people know that automation is difficult to maintain. There is only one reason for maintenance: demand change. Whether it is caused by page modification, logic modification or underlying code change, it can be attributed to requirements.

Each maintenance is usually divided into the following steps:
1. Run all use cases and get the wrong use cases.
2. Analyze the wrong use cases and find out the use cases caused by this modification.
3. Run again after modifying the use case and confirm that there is no problem.
4. Repeat step 3 until all error cases have been modified.
5. Run all the use cases again to confirm that there is no problem and merge the code. If there is a problem, repeat steps 3 and 4.

The above steps are very time-consuming. Usually, the automation team of one or two people, even if it is full-time, will spend most of the time on maintenance when the use cases accumulate to 500 or 600, not to mention many are not full-time.

It’s time-consuming and labor-consuming. If the effect can be guaranteed after modification, it’s actually reluctantly acceptable. However, unfortunately, the above maintenance actions are usually found to have problems when we officially run the use case report, which also leads to the passing rate in the report we send out from high to low. When there is no UI modification, the wind is calm and the waves are calm. Once there is a modification, there will be a chicken feather on the ground.

The instability of the passing rate will lead to a terrible thing: distrust.

Imagine that you are a test leader or development or business testing classmate. When an automated test report is sent, click open an error case and find that it is caused by UI modification. Click open the second one or finally find an error caused by other reasons in a pile of failed cases caused by non maintenance. Check with you and find that it is a network problem. What will be the mentality of people who read this report?

After this many times, people who care about this report will not remember what you look like when your success rate is 95%, but only when your success rate is 80%, and come to the conclusion that this thing is useless.

In fact, this is still a good situation. After all, it is very good that the passing rate can reach 95%, while the UI automation passing rate of most companies in the market has never reached 95%, or even 80%. Who can show such results? Who lost it?

Conclusion: maintenance takes effort and effort, and the result is still untrusted.

The above three points are the reasons why UI automation is still weak after so many years. At present, it is a thing that the actual input is 100 points and the output is only 90 points, and then it needs to continue to invest higher and higher costs to continue to maintain 90 points of output. But it is regarded as a once and for all solution with 50 points of actual input and 150 points of output by people who don’t know or pretend not to know. This has attracted a large group of people to enter this field. After reading a piece of pseudo code, they say they can. After writing a complete use case, they say they are proficient. If they are working on a half hanging project in a company, they say they are experts.

So the problem is, it doesn’t seem so beautiful. Shall we do it? There are so many difficulties. Shall we do it? The follow-up investment seems to be a bottomless pit. Should we do it?

Recommended Today

Is there a loop in the ring array

Source: leetcodeLink:https://leetcode-cn.com/problems/circular-array-loop Title Description: There is a ring array nums without 0. Each nums [i] represents the number of subscripts that the character with subscript i should move forward or backward:If num [i] is a positive number, move num [i] step forwardIf num [i] is negative, move num [i] step backwardBecause the array is circular, […]