The length of the article is long, about 20 minutes after reading, it is recommended to collect and read, and there will be harvest after reading. Welcome to like your attention
Link to the original text: https://www.softwaretestinghelp.com/types-of-software-testing/
How many types of software testing are there?
As testers, we know many different types of software testing, such as functional test, non functional test, automatic test, agile test, and their various sub types. Although we will encounter many types of testing in our testing process, or have heard of some types of testing, few people dare to say that they are proficient in all types of testing
Each test type has its own characteristics, advantages and disadvantages. So I write this article to popularize the most commonly used test types today
The article is free translation, and on the basis of the original text deduction and expansion
Different types of software testing
The following is a list of common types of software testing
Function test type:
Beta / acceptance testing
Non functional test type:
Take a look at the details of these test types
As the name implies, a / B test is to prepare two (A / b) or more versions, let different users access these versions randomly, collect user experience data and business data of each group, and finally analyze and evaluate the best version for formal adoption. As shown above, Google uses the A / B test to determine whether the navigation should be red or blue.
- Alpha testing
Alpha test this is a very common test type in software engineering. Its goal is to identify as many problems and defects as possible before it is released to the market or delivered to users.
Alpha testing is usually done at the end of development and before beta testing. During the testing process, developers may be driven to make some minor design changes. Alpha testing is generally conducted on the developer website, that is, only open to developers or internal users. Generally, internal virtual user environment can be created for such tests.
Generally, large software projects have a standardized software version cycle
Pre alpha: sometimes software releases pre alpha before alpha or beta. Compared with alpha and beta, this is an incomplete version
Alpha: the function of alpha version is not perfect and needs further testing. The alpha version is usually sent to the software development organization or software testers in a certain group for internal testing.
Beta: generally, the beta version will contain all functions, but there may be some bugs that need debugging feedback. Beta version is the earliest software version that is publicly available and is tested by the public (usually third-party developers and amateur players outside the company).
Release candidate (RC): release the candidate version. If there is no problem, it can be released to the official version. This version contains complete and stable functions
To give a typical example, the release plan of IOS 13 that recently made me a bit miserable:
June 3: IOS 13 beta 1 and first look at WWDC 2019 ා – > WWDC can be installed, which is equivalent to pre alpha or alpha stage
June 17: iOS 13 beta 2 launched for developers
June 24: IOS 13 public beta release date for innovative testers
July 3: iOS 13 developer beta 3 launch with some new features
July 8: iOS 13 public beta 2 release date
Early September 2019: IOS 13 golden master (final dev beta) ා – – – – in early September, the final beta version should be released, which is equivalent to entering the RC stage
Mid September 2019: IOS 13 like to launch with new 2019 iPhones
Now many open-source projects have weakened the waterfall software version cycle and become a continuous and normalized behavior, such as Firefox:
2) Acceptance testing
Acceptance test is usually the last test operation before software deployment, also known as delivery test. It is performed by the end customer. They will verify whether the end-to-end system process meets the business requirements, and whether the function meets the requirements of the end-user. The customer will accept the software only if all the features and functions run as expected
This is the final stage of testing. After acceptance testing, the software will be put into production. So it is also called user acceptance test (UAT)
For example, acceptance testing is equivalent to receiving express delivery. The package is software, you are the customer, and you are the acceptance party. If the goods do not meet your requirements, they will be returned.
Ad hoc testing
Ad hoc Chinese should be understood as temporary. As the name suggests, this kind of test is carried out on a temporary basis, sometimes called random test. That is, there is no reference test case, no plan and documentation for the test. The purpose of ad-hoc testing is to find out the application defects and problems by executing arbitrary processes or functions
Ad hoc testing is an informal method that can be performed by anyone in the project. Although it is difficult to identify defects without test cases, some defects found during ad-hoc testing may not be identified by existing test cases, that is to say, it is generally used to find “unexpected” defects
The purpose of accessibility testing is to determine whether software or applications are available to people with disabilities. Disability refers to the deaf, color blind, the mentally disabled, the blind, the elderly and other disabled groups. Various checks are performed here, such as font size tests for visual disabilities, color and contrast tests for color blindness, and so on.
Different platforms and application types have different support for accessibility. For example, IOS pays more attention to accessibility than other operating systems, while foreign countries pay more attention to accessibility than domestic ones.
5) Beta testing
Beta testing has been mentioned in the alpha test above. Beta testing is a formal software testing type, which is executed by the customer in the real application environment before the product is released to the market or the actual end user.
The purpose of beta testing is to ensure that there are no major failures in the software or product and meet the business needs of end users. Beta testing is only passed when the customer accepts the software.
Typically, such testing is done by the end user or someone else. This is the final test done before the application is released for commercial use. In general, beta releases of software or products are limited to a specific number of users in a specific region.
So the end users will feedback some problems to the company after they actually use the software. The company can take necessary measures before the full release.
Beta testing may also be repeated several times before the official release
Back end testing
The data input by the front-end application is generally stored in the database, so this kind of test for the database is called database test or back-end test. There are different databases in the market, such as SQL server, MySQL and Oracle. Database testing involves table structure, schema, stored procedure, data structure, etc.
Generally, GUI is not involved in back-end testing. Testers can connect to the database directly by some means, so they can easily run some database requests to verify the data. Through the back-end test, we can find some database problems, such as data loss, deadlock, data corruption. These problems are critical to be fixed before the system is put into production
Browser compatibility testing
This is a subtype of compatibility test, which is performed by the test team. Browser compatibility test is mainly aimed at web applications, which is used to ensure that the software can run in different browsers or operating systems; or to verify whether the web application supports running on all versions of the browser, so as to determine the final compatibility scope of the application
Browser compatibility testing is a hole that front-end developers can’t get around.
We have many strategies to deal with browser compatibility, such as progressive enhancement or elegant degradation, and the formulation of browser compatibility specifications;
In order to smooth the differences between browsers, we will use various feature detection tools (modernizr), as well as various Polyfill (CSS normaliz, Polyfill / shim, CSS autoprefix);
Of course, in order to test the cross browser compatibility, we also need some auxiliary tools, such as browserstack. For our small teams, we can only manually test the next batch of portable (portable browsers are isolated from each other at runtime, so there will be no conflict problems such as configuration files).
Backward compatibility testing
Backward compatibility test is used to verify whether the newly developed or updated software can run in the old version environment.
For example, the backward compatibility test will check whether the new version of the software can correctly handle the file format created by the old version. For example, whether the new version of office 2016 can open files created in 2012.
Similarly, you can check whether the new version is compatible with the data tables, data files, data structures, and configuration files created by the old version of the software.
Any software update should run well on top of previous versions
Black testing box
Black box testing does not consider the internal system design of the software, it tests based on requirements and functions, and only cares about the input / output and functional process of the system.
In other words, black box testing is aimed at software interface, function and external structure from the user’s point of view, without considering the internal logic structure of the program
There are many subclasses under black box test, such as integration test, system test and most non functional tests
Boundary value testing
Boundary value test: test the behavior of the application in boundary level. Many boundary condition developers are hard to be considerate, so there is a special test type to verify this situation
The boundary value test checks for defects when the application is at the boundary value. Boundary value test is usually used to test numbers in different ranges. Each range has an upper and lower boundary, and boundary test is used to test these boundary values.
For example, if the number range is 1-500, the boundary value test will verify on these values: 0, 1, 2, 499, 500, 501
This is a subtype of white box test, which is implemented in unit test. As the name implies, branch test means that the test should cover all conditional branches of program code to avoid missing defects. Branch coverage is one of the indicators of unit test coverage
Comparative testing, comparing the advantages and disadvantages of the product with the old version or similar products
For example, when evaluating a mobile phone or other digital products, the digital evaluation column like Wang Ziyi will generally compare with the products of friends horizontally, and sometimes with the products of the previous generation vertically
Another typical example is the comparison with industry leaders. For example, when we do im, we often compare it with wechat: “how can your app start so much slower than wechat?”
This is a large class, compatibility testing is used to verify the behavior of applications in different environments, web servers, hardware, network conditions. Compatibility testing ensures that the software can run in different configurations, databases, browsers, and versions. Compatibility testing is carried out by the test team
Component testing (this component is not a GUI component, it may be better to understand the combination test), which is also known as module testing, and is generally executed by developers after completing the unit test. The purpose of component testing is to find the defects of multiple functions after they are connected.
Component testing can be large or small, small to function level or class level combination, large to several separate page, module, subsystem combination. Taking a front-end example, it belongs to component testing to combine multiple page routes and test their process jump.
End to end testing
End to end testing is also a type of black box testing, similar to system testing. End to end testing simulates real users to test applications in simulated, complete and real application environments. For example, applications will interact with databases, use network communication, or interact with other hardware, applications and systems when appropriate End to end means from one endpoint to another, so end to end testing focuses on testing the coordination between modules.
When the application is a distributed system or needs to cooperate with other external systems, end-to-end testing plays a very important role. It can check comprehensively to ensure that the software can interact accurately in different platforms and environment products. End to end testing has the following purposes:
Ensure good coordination between application and external system. For the front end, it is to ensure good coordination between the page and the back end
Check all system flows from the source system to the target system
Validate requirements from an end user perspective
Identification in heterogeneous environment
There are also many automated end-to-end testing tools in the front-end, such as nightwatch, through which users can simulate the operation of the page, so as to check whether the whole application process is normal and meets the requirements
Because they are similar to system tests, they are often compared
- Equivalence partitioning
Equivalence partitioning is a black box testing technique. Through equivalence partitioning, all input data can be reasonably divided into multiple groups. We only need to take one data in each group as the test input condition, so we can achieve better test results with a small number of representative test data
So the purpose of this test is to remove the repeated use cases in the specified group without causing defects and simplify the test work
For example, a program application accepts values between – 10 and + 10, and can be divided into three groups: 0, negative value and positive value by using equivalent partition method. The next test only needs to test one member from the three groups, instead of testing each member from – 10 to + 10
- Example testing
It means real-time testing. Example testing includes the real-time scenario, it also involves the scenarios based on the experience of the testers.
Instance testing means real-time testing. Instance testing includes real-time scenarios and scenarios based on testers’ experience.
This test type is not particularly understood here, so post the original. Tell me what you know
- Exploratory testing
Exploratory testing is a bit like ad hoc testing. Exploratory testing is an informal test conducted by the test team. The purpose of this test is to explore the application and find the defects in the application. Like exploration, there is a certain probability that significant defects, which may even lead to system failure, are discovered during the test
During exploratory testing, it is recommended to track and record the testing process and activity records before starting the process to facilitate bug recurrence
No documentation and test cases are required to explore tests
- Functional testing
Test is a kind of function oriented test, which focuses on the function of test box.
Functional testing is relative to non functional testing. Functional testing needs to care about function or business and needs high degree of business coupling. Non functional testing is universal, such as stress testing and load testing. These tests are supported by universal tools and do not need or few customized operations
Graphical user interface (GUI) testing
The purpose of GUI testing is to validate the GUI against business requirements. In the detailed design documents and GUI models (UI design documents), the GUI expected by the application is generally mentioned
Common GUI tests include testing the size of the buttons and input fields displayed on the screen, the alignment rules of all text, tables or content in the form, and so on. If the team has a UI design specification, it also verifies that it meets the design specifications
Gorilla testing is a type of testing performed by testers and sometimes by developers. In gorilla testing, a module or function in the module is thoroughly and strictly tested. The original text does not say the essence of gorilla testing. Gorilla testing will repeat “hundreds of times” on a function or module. Humans simply can’t stand such a test method. Therefore, another nickname of gorilla test is “frustrating testing.”
The purpose of this test is to check the robustness of the application
Happy path testing
The goal of optimistic route testing is to successfully test the application on the normal process. It doesn’t take into account negative or unusual situations. The focus is only to verify that the application generates the desired output under the conditions of valid and legal input. For example, bank payment only considers the normal state of account money
Incremental integration testing
Incremental integration testing is a bottom-up approach, which integrates applications for continuous testing as soon as new features are added. Application functions and modules should be sufficiently independent to be tested separately. This is usually done by programmers or testers.
Install / uninstall testing
Installation and uninstall test is a complete / partial test of installation, upgrade, uninstall and rollback on different operating systems under different hardware or software environments. It is commonly used in desktop applications
Integration test is to verify the combined function after integrating all modules. Modules are usually code modules, single applications, client and server applications on the network, etc.
Integration test is generally after unit test, so unit test is the basis of integration test, and integration test without unit test is not reliable. So the simplest form is: “combine two tested units into a component and test the interface between them.”. That is to say, on the basis of unit test, integration test combines the independent units in unit test to verify their coordination. The combined component is a new “unit”. In this way, the test is gradually combined to form a complete application.
This type of testing is commonly used in B / s software and distributed systems.
- Load testing
It is a non functional test. The purpose of load test is to check how much load the system can bear without reducing performance, or to determine the maximum workload.
Load testing helps to find out the maximum capacity of the system under a specific load and any reason for software performance degradation. You can use JMeter, LoadRunner, webload, silk executor and other tools to perform load tests.
Load testing is often associated with performance testing, stress testing, stability testing and so on. As shown in the figure above (from the performance white paper of Taobao), TPS (transformation per second) refers to the number of transactions or transactions that can be processed by the system per second; server resource refers to the possession of system resources
Performance test is mainly between a-b. in the early stage of system design, an expected goal will be planned, for example, given a resource ax, point a is the performance expectation. In other words, given the fixed resource ax, if TPS can reach point a or even higher, it means that the system performance is up to or better than expected. Through the performance test, we can verify whether the processing capacity of the system has met the expectation
Load test. Located between b-c. Increase concurrent requests to the system until one or more indicators of the system reach a safe threshold, as shown in the figure above, C is the so-called maximum load. If the request pressure is increased later, the processing capacity of the system will not be improved and the return will be reduced. The maximum safe load value of the system can be obtained through the pressure test
Pressure test. Between c-d. In the case of exceeding the safe load, continue to increase the pressure on the system until it reaches the collapse point, i.e. D. in the figure above, the maximum bearing capacity of the system can be obtained through the pressure test
Stability test. Between A-D. At different points of a, B, C and D (representing specific hardware, software and network environment), let the system run for a long time, and check the stability of the system under different conditions.
It is also recommended to read this book
Monkey testing is conducted by testers, that is, they regard themselves as monkeys and input and operate at will without any knowledge background or understanding of the application.
The goal of the monkey test is to check whether an application or system crashes by providing random input values / data. The monkeys were randomly executed, there were no test cases, and there was no need to understand the full functionality of the system
Variation test (or variability test) is a white box test, which is the opposite of unit test.
Generally, the idea of unit test is to verify whether the code is valid and reliable through test cases, while mutation test is the reverse. It first changes the source code of one program, and then runs unit test. If the unit test passes, it may indicate that the test case has no effect, or the test case does not cover the code mutation
So mutation testing can in turn verify whether your test case is valid, and it can help us find out some potential errors that cannot be prevented by the current test
On the contrary, pessimistic test and optimistic route test require the tester to have a “break” attitude, consider all kinds of abnormal situations, and use all kinds of evil, malicious and illegal operations to test the system. Pessimistic tests use incorrect data, invalid data, or input for validation. It verifies that the system can identify exceptions and run as expected.
Non functional testing
Each large organization has an independent team, often referred to as a non functional testing (NFT) team or performance team.
Non functional testing involves testing non functional requirements, such as load testing, stress testing, security, capacity, recovery testing, etc. the goal of NFT testing is to ensure that the response time of software or application meets the business requirements.
For example, loading any page or system should not take too long, and should be in good working condition during peak load.
This term is often used interchangeably with “stress” and “load” tests. Performance testing is used to check whether the system meets the performance requirements. It uses different performance and load tools to perform this test.
The scope of performance test is relatively large. In a broad sense, performance test includes load test, pressure test, stability test, capacity test and so on mentioned above. In a narrow sense, performance test refers to whether the test system can achieve the expected value under the specific resource conditions, that is, baseline test
Summarize the types of performance tests:
Baseline test: under the given resources, test the best performance and use it as a reference ‘baseline’ for subsequent measurement. Note that there is a difference between the baseline test and the benchmark test. So, the benchmark is what you want to achieve, such as the world record of 100 sprint, and the baseline is your score.
Load test: measure the performance of the system under the expected peak production load. Load testing has been outlined above
Stability test: measure the stability of the system for a long time under the specified load
Stress test: test system performance under extreme conditions
Recovery testing is used to verify the extent of a crash or disaster recovery in an application or system. It determines whether the system can continue to run after a disaster.
For example, when an application receives data through a network cable, it suddenly disconnects the network cable. After a period of time, it plugs in the network cable, and the system should start to recover the data lost due to the network cable unplugging
After modifying any module or function, testing the application as a whole is called regression testing. The purpose of regression testing is to verify the integrity of software after the original function changes
Some people think that regression testing is regression testing, which means repeating all or part of the same testing work before, which is not unreasonable. Moreover, it is not uncommon that the whole body is triggered by partial modification, which is the purpose of regression testing
Because it is difficult to cover all systems in regression testing, it is usually best to use automated testing tools for these class tests. For example, after modifying the code, run unit test to ensure that other software units are not affected.
In the front end, component snapshot testing and some CSS UI tests belong to the regression test type. Their principles are compared with the results generated in the last test to ensure no unexpected changes:
35) risk based testing (RBT)
In risk-based testing, the function or requirement will be tested according to its priority. Risk based testing will give priority to testing highly critical functions, because these functions have the greatest impact on the business or the probability of failure is very high. The priority is determined by the business requirements, so once the priority is set for all functions, the high priority functions or test cases should be executed first, and then the low priority functions. Low priority functions can be tested when there is plenty of time, or not.
Risk based testing should be performed when “there is not enough time to test the entire application, but the software needs to be delivered on time”. It usually needs to be discussed and approved by customers and senior management
- Integrity testing
Integrity testing is used to determine whether a new software version can start formal testing. If a software version should crash at the beginning of use, it indicates that the system is not stable enough and there is no need for further testing. This situation should be called back to the development to avoid wasting time
Take our company as an example:
- List item
In the software design phase, the test team will write smoke test cases;
- List item
The development team will run its own smoke case before submitting the version to the test to ensure there is no major failure;
- List item
After the version is submitted to the test team, the test team will run the integrity test first to check whether there are major bugs that affect the test process, and if so, they will return to development
- List item
If the integrity test is passed, the smoke test will be carried out, and if the smoke test fails, the development will be called back immediately.
- List item
Only after the integrity test and smoke test are successfully passed can the formal test phase be entered.
One of the purposes of this is to reduce the workload of the test team, because they have to connect the test tasks of multiple development teams.
- Security testing
Security is also a huge discipline, and knowledge is updated every day, so security testing is generally performed by special security teams, who use various hacker means to conduct penetration testing on the system.
The purpose of security testing is to ensure that the application or website is free from internal and external threats. This test includes preventing malicious programs and viruses; verifying the security of authorization and authentication processes.
It also checks how the software reacts to any hacker attacks and malicious programs, and how to maintain the software to protect data security after being attacked by hackers
Smoke test: every time the development team submits a new build, the software test team will verify the build first and ensure that there are no major problems. If there are major problems, they will be directly called back to the development team
How to understand smoke test in general? This belongs to the hardware industry. After a hardware or a hardware component is changed or repaired, it directly powers up the device. If there is no smoke, the component passes the test. For example, if you power up the Samsung Note7, if it doesn’t explode, it means that it has passed the “smoke test” (it’s not easy to be a mobile phone tester, it’s easy to be life-threatening)?
The test team will not perform further detailed tests until the build is stable. The smoke check checks for a stop defect in the build, which prevents the test team from further detailed testing. That is, if testers find that the main functionality doesn’t work, they reject the build and return it to the development team.
Smoke testing is usually performed prior to regression testing or other detailed testing
Static testing is a bit like code review, which is executed without executing any code (that is, without running the application). It involves inspection, review, and walkthrough. For example, checking code syntax, naming conventions, and project organization.
Static testing is not only applicable to code, but also to test cases, test plans and design documents. If defects are found in the static testing phase, the cost of defects can be minimized. For example, it is better to find problems in the design stage than in the development stage or even in the production environment
For example, static testing may include:
Use the lint tool to check the specification of the program. The related tools are eslint, tslint, stylus, etc. even typescript, these type checkers can be included in this category
Code review. There are some problems that cannot be covered by lint tools, such as code logic, exception capture, project organization, memory leak, etc., which need to be reviewed manually
Check whether the code is consistent with the design, whether it meets the software requirements, summary and detailed design, which can not only see the code problems, but also find out whether the requirements or design are correct earlier.
- Stress testing
Through the stress test, it simulates the failure mode and time of the system under the pressure beyond its specification, and finds out the collapse point of the system. This test is executed under high load conditions, such as accessing data exceeding the capacity limit, executing complex database query, continuously and violently inputting to the system or loading into the database.
- System testing
System testing is carried out on a complete integrated system, that is to say, the system test is generally conducted after the integration test. After the integration test, the system becomes a whole. On this basis, the system test verifies whether the system meets the business requirements in the real running environment. This is a black box test, based on the overall requirements specification, covering all the combined parts of the system.
System testing is not a specific testing technology, but a testing stage. There will be many kinds of tests in this stage, and the work of the test team of the general company will focus on this part. Generally including:
Function test: that is to say, test whether the system meets the business requirements from the whole
Various non functional tests: such as recovery test, performance test, stress test, safety test, etc.
Summarize the purpose of system testing:
Ensure that the application as a whole works well
Ensure applications meet business requirements
Ensure the application can run well in the real environment. For example, some non functional tests are carried out to verify the robustness of the system
In fact, system testing is very similar to the end-to-end testing mentioned above. They require the system to be tested as a whole. You can simply expand the comparison
- Unit testing
Testing independent software units or modules is called unit testing. It is usually done by developers, not by testers, because it requires a detailed understanding of the internal programming and code.
Unit testing is the type of test most closely related to our developers. Its test object is software unit. A software unit can be a function / method, a class, a GUI component, etc.
This is a white box test, so it needs to be done by the developer himself, because only the developer knows the internal implementation of the unit. Unit tests generally use test coverage to verify the completion of unit tests
The common unit test tools on the front end are jest, mocha, jasmine, etc. the following is a typical BDD style unit test organization:
43) usability testing
Usability testing is used to test the user friendliness of an application. It will verify that the new user can easily understand the application process. If the user is in trouble, the tester should record and provide help. It can be considered that the usability test is to check the navigation of the system
Vulnerability testing involves identifying vulnerabilities in software, hardware and networks. If the vulnerability is vulnerable to attack, or susceptible to viruses and worms, hackers or malicious programs can control the system.
Therefore, it is necessary to check these systems for vulnerabilities before putting them into production environment.
Capacity testing is a non functional test performed by a performance testing team. The capacity test checks the system behavior and response time when the application encounters large amounts of data. This large amount of data may affect the performance of the system and the speed of processing time.
White box testing
White box testing, also known as glass box testing, structural testing, logic driven testing or code based testing, is based on the internal logic of application code. That is, testers should know how the internal software and code work, and test all the logical paths. The unit test and static test mentioned above are typical white box tests. Basically, white box tests can be equivalent to unit tests
Logical path includes statement coverage, decision coverage, condition coverage, decision / condition coverage, condition combination coverage and path coverage
The above mentioned software test types are only a part of the test. In fact, there are more than 100 test types. (718897738) welcome to exchange software tests together. The group provides the latest free video + tools to download. But not all test types will be used by all projects, so I just list some common software test types here.
In addition, different organizations may have different definitions or processes, but the basic concepts are the same everywhere. As the project, requirements, and scope change, these test types, processes, and implementation methods evolve.