The length of the article is long, about 20 minutes after reading, it is recommended to collect and read, and there will be harvest after reading. Welcome to like your attention
Link to the original text: https://www.softwaretestinghelp.com/types-of-software-testing/
How many types of software testing are there?
As testers, we understand many different types of software testing, such as functional testing, non functional testing, automated testing, agile testing, and their various sub types. Although we have many test types in our testing process, or have heard of some test types, few people dare to say that they are proficient in all test types
Each test type has its own characteristics, advantages and disadvantages. So I’m going to write this article about the most commonly used test types today
The article is free translation, and on the basis of the original text deduction and expansion
Different types of software testing
The following is a list of common types of software testing
Function test type:
Beta / acceptance testing
Non functional test type:
Take a look at the details of these test types
As the name suggests, the A / B test is to prepare two (A / b) or more versions, let different users randomly access these versions, collect user experience data and business data of each group, and finally analyze and evaluate the best version, and officially adopt it. As shown above, Google uses the A / B test to determine whether the navigation should be red or blue.
- Alpha testing
Alpha test this is a very common test type in software engineering. Its goal is to identify as many problems and defects as possible before it is released to the market or delivered to users.
Alpha testing is usually done at the end of development and before beta testing. During the testing process, developers may be driven to make some minor design changes. Alpha testing is generally conducted on the developer website, that is, only open to developers or internal users. Generally, internal virtual user environment can be created for such tests.
Generally, large software projects have a standardized software version cycle
Pre alpha: sometimes software releases pre alpha before alpha or beta. Compared with alpha and beta, this is an incomplete version
Alpha: the function of alpha version is not perfect and needs further testing. The alpha version is usually sent to the software development organization or software testers in a certain group for internal testing.
Beta: generally, beta version includes all functions, but there may be some bugs that need debugging feedback. Beta version is the earliest software version that is publicly available and is tested by the public (usually third-party developers and amateur players outside the company).
Release candidate (RC): release the candidate version. If there is no problem, the official version can be released. This version contains complete and stable features
A typical example is the release plan of IOS 13, which has made me a little miserable recently
June 3: IOS 13 beta 1 and first look at WWDC 2019 ා – > WWDC can be installed, which is equivalent to pre alpha or alpha stage
June 17: iOS 13 beta 2 launched for developers
June 24: IOS 13 public beta release date for innovative testers
July 3: iOS 13 developer beta 3 launch with some new features
July 8: iOS 13 public beta 2 release date
Early September 2019: IOS 13 golden master (final dev beta) ා – – – – in early September, the final beta version should be released, which is equivalent to entering the RC stage
Mid September 2019: IOS 13 like to launch with new 2019 iPhones
Nowadays, many open source projects have desalinated the waterfall software version cycle and become a continuous and normalized behavior, such as Firefox:
2) Acceptance testing
Acceptance test is usually the last test operation before software deployment, also known as delivery test. It is performed by the end customer. They will verify whether the end-to-end system process meets the business requirements, and whether the function meets the requirements of the end-user. The customer will accept the software only if all the features and functions run as expected
This is the final stage of testing. After acceptance testing, the software will be put into production. So it is also called user acceptance test (UAT)
For example, acceptance testing is equivalent to receiving express delivery. The package is software, you are the customer, and you are the acceptance party. If the goods do not meet your requirements, they will be returned.
Ad hoc testing
Ad hoc in Chinese should be understood as temporary. As the name suggests, this type of testing is done on a temporary basis, sometimes referred to as random testing. That is, there is no reference test case, no plan and documentation for the test. The purpose of ad hoc testing is to find out application defects and problems by executing arbitrary processes or arbitrary functions
Ad hoc testing is an informal method that can be performed by anyone in the project. Although it is difficult to identify defects without test cases, sometimes defects found during ad hoc testing may not be identified by existing test cases, that is, they are generally used to detect “unexpected” defects
The purpose of accessibility testing is to determine whether software or applications are available to people with disabilities. Disability refers to the deaf, the color blind, the mentally handicapped, the blind, the elderly and other disabled groups. Various checks are performed here, such as font size tests for visual disabilities, color and contrast tests for color blindness, and so on.
Different platforms and application types have different support for accessibility. For example, IOS pays more attention to accessibility than other operating systems, while foreign countries pay more attention to accessibility than domestic ones.
5) Beta testing
Beta testing has been mentioned in alpha testing above. Beta testing is a formal type of software testing, which is performed by customers in a real application environment before the product is released to the market or actual end users.
The purpose of beta testing is to ensure that there are no major failures in the software or product and meet the business needs of end users. The beta test passes only when the customer accepts the software.
Typically, such testing is done by the end user or someone else. This is the final test done before the app is released for commercial use. Generally, beta releases of software or products are limited to a specific number of users in a specific region.
So the end users will feedback some problems to the company after they actually use the software. The company can take necessary measures before the full release.
Beta testing may also be repeated several times before the official release
Back end testing
The data input by front-end applications is generally stored in the database, so this kind of test for database is called database test or back-end test. There are different databases in the market, such as SQL server, MySQL and Oracle. Database testing involves table structure, schema, stored procedure, data structure, etc.
Generally, GUI is not involved in back-end testing. Testers can connect to the database directly by some means, so they can easily run some database requests to verify the data. Through the back-end test, we can find some database problems, such as data loss, deadlock, data corruption. It is very important to fix these problems before the system is put into production environment
Browser compatibility testing
This is a subtype of compatibility testing, which is performed by the test team. Browser compatibility testing is mainly aimed at web applications and is used to ensure that the software can run in different browsers or operating systems, or to verify whether the web application supports running on all versions of the browser to determine the ultimate compatibility scope of the application
Browser compatibility testing is a hole that front-end developers can’t get around.
We have many strategies to deal with browser compatibility, such as progressive enhancement or elegant degradation, and the formulation of browser compatibility specifications;
In order to smooth the differences between browsers, we will use various feature detection tools (modernizr), as well as various Polyfill (CSS normaliz, Polyfill / shim, CSS autoprefix);
Of course, in order to test the cross browser compatibility, we also need some auxiliary tools, such as browserstack. For our small teams, we can only manually test the next batch of portable (portable browsers are isolated from each other at runtime, so there will be no conflict problems such as configuration files).
Backward compatibility testing
Backward compatibility test is used to verify whether the newly developed or updated software can run in the old version environment.
For example, the backward compatibility test will check whether the new version of the software can correctly handle the file format created by the old version. For example, whether the new version of office 2016 can open files created in 2012.
Similarly, you can check whether the new version is compatible with the data tables, data files, data structures, and configuration files created by the old version of the software.
Any software update should run well on top of previous versions
Black box testing
Black box testing does not consider the internal system design of the software, it tests based on requirements and functions, only concerns about the input / output and function flow of the system.
In other words, black box testing is aimed at software interface, function and external structure from the user’s point of view, without considering the internal logic structure of the program
There are many subclasses under black box testing, such as integration testing, system testing, and most non functional testing
Boundary value testing
Boundary value testing, which tests the behavior of an application at boundary level. Many boundary condition developers are hard to be considerate, so there is a special test type to verify this situation
The boundary value test checks for defects when the application is at the boundary value. Boundary value test is usually used to test numbers in different ranges. Each range has an upper and lower boundary, and boundary test is used to test these boundary values.
For example, if the number range is 1-500, the boundary value test will verify on these values: 0, 1, 2, 499, 500, 501
This is a subtype of white box testing, which is implemented in unit testing. As the name suggests, branch testing means that the test should cover various conditional branches of program code to avoid missing defects. Branch coverage is one of the indicators of unit test coverage
Comparative testing, comparing the advantages and disadvantages of the product with the old version or similar products
For example, when evaluating a mobile phone or other digital products, it generally compares with the products of friends horizontally, and sometimes vertically with the products of the previous generation
Another typical example is the comparison with industry leaders. For example, when we do im, we often compare it with wechat: “how can your app start so much slower than wechat?”
This is a large class, compatibility testing is used to verify the behavior of applications in different environments, web servers, hardware, network conditions. Compatibility testing ensures that the software can run in different configurations, different databases, different browsers, and their different versions. Compatibility testing is carried out by the test team
Component testing (this component is not a GUI component, it may be better to understand the combination test), which is also known as module testing, and is generally executed by developers after completing the unit test. The purpose of component testing is to find the defects of multiple functions after they are connected.
Component testing can be large or small, small to function level or class level combination, and large to several individual page, module, subsystem combination. Taking a front-end example, it belongs to component testing to combine multiple page routes and test their process jump.
End to end testing
End to end testing is also a type of black box testing, similar to system testing. End to end testing simulates real users to test applications in simulated, complete and real application environments. For example, applications will interact with databases, use network communication, or interact with other hardware, applications and systems when appropriate End to end means from one endpoint to another, so end-to-end testing focuses on testing the coordination between modules.
When the application is a distributed system or needs to cooperate with other external systems, end-to-end testing plays a very important role. It can be checked comprehensively to ensure that the software can interact accurately in different platforms and environments. End to end testing has the following purposes:
Ensure good coordination between application and external system. For the front end, it’s about ensuring good coordination between the page and the back end
Check all system flows from the source system to the target system
Validate requirements from an end user perspective
Identify problems in heterogeneous environments
There are also many automated end-to-end testing tools in the front-end, such as nightwatch, through which users can simulate the operation of the page, so as to check whether the whole application process is normal and meets the requirements
Because they are similar to system tests, they are often compared
- Equivalence partitioning
Equivalence partitioning is a black box testing technique. Through equivalence partitioning, all input data can be reasonably divided into multiple groups. We only need to take one data in each group as the test input condition, so we can achieve better test results with a small number of representative test data
Therefore, the purpose of this test is to remove the repeated use cases in the specified group and simplify the testing work without causing defects
For example, a program application accepts values between – 10 and + 10, and can be divided into three groups: 0, negative value and positive value by using equivalent partition method. The next test only needs to test one member from the three groups, instead of testing each member from – 10 to + 10
- Example testing
It means real-time testing. Example testing includes the real-time scenario, it also involves the scenarios based on the experience of the testers.
Instance testing means real-time testing. Instance testing includes real-time scenarios and scenarios based on testers’ experience.
I don’t really understand this type of test here, so I’ll post the original. Tell me what you know
- Exploratory testing
Exploratory testing is a bit similar to ad hoc testing. Exploratory testing is informal testing conducted by test teams. The purpose of this test is to explore the application and find defects in the application. Like exploration, there is a certain probability that significant defects, which may even lead to system failure, are discovered during the test
During exploratory testing, it is recommended to track and record the testing process and the activity record before starting the process to facilitate the recurrence of bugs
Exploratory testing does not require any documentation and test cases
- Functional testing
Functional testing is a large class, also known as behavioral testing. Functional testing will ignore the internal implementation and focus on the output of components, in order to verify whether it meets the requirements. This is a kind of black box test type oriented to functional requirements.
Functional testing is relative to non functional testing, functional testing needs to care about function or business, and needs high degree of business coupling; while non functional testing is general, such as stress testing and load testing, which are supported by common tools, and do not need or have little customized operation
Graphical user interface (GUI) testing
The purpose of GUI testing is to validate GUI according to business requirements. In the detailed design documents and GUI models (UI design documents), the GUI expected by the application is generally mentioned
Common GUI tests include testing the size of the buttons and input fields displayed on the screen, the alignment rules of all text, tables or content in the form, and so on. If the team has a UI design specification, it also verifies that it meets the design specifications
Gorilla testing is a type of test performed by testers and sometimes by developers. In gorilla testing, a module or function in the module is thoroughly and strictly tested. The original text does not say the essence of gorilla testing. Gorilla testing will repeat “hundreds of times” on a function or module. Humans simply can’t stand such a test method. Therefore, another nickname of gorilla test is “frustrating testing.”
The purpose of this test is to check the robustness of the application
Happy path testing
The goal of optimistic route testing is to successfully test the application on the normal process. It doesn’t take into account negative or unusual situations. The focus is only on verifying that the application generates the desired output under the conditions of valid and legal input. For example, bank payment only considers the normal state of account money
Incremental integration testing
Incremental integration testing is a bottom-up approach, which integrates applications for continuous testing as soon as new features are added. Application functions and modules should be sufficiently independent to be tested separately. This is usually done by programmers or testers.
Install / uninstall testing
Installation and uninstall test is a complete / partial test of installation, upgrade, uninstall and rollback on different operating systems under different hardware or software environments. It is commonly used in desktop applications
Integration testing refers to the integration of all modules and the verification of the combined functions. Modules are usually code modules, single applications, client and server applications on the network, etc.
The integration test is generally after the unit test, so the unit test is the basis of the integration test, and the integration test without unit test is unreliable. So the simplest form is: “combine two tested units into a component and test the interface between them.”. In other words, on the basis of unit test, the integration test combines the independent units in the unit test to verify their coordination, and the merged component is a new “unit”. In this way, the test is gradually merged to form a complete application program.
This type of testing is commonly used in B / s software and distributed systems.
- Load testing
It is a non functional test. The purpose of load test is to check how much load the system can bear without reducing performance, or to determine the maximum workload.
Load testing helps to find out the maximum capacity of the system under a specific load and any reason for software performance degradation. You can use JMeter, LoadRunner, webload, silk executor and other tools to perform load tests.
Load testing is often associated with performance testing, stress testing, stability testing and so on. As shown in the figure above (from the performance white paper of Taobao), TPS (transformation per second) refers to the number of transactions or transactions that can be processed by the system per second; server resource refers to the possession of system resources
Performance testing. It mainly lies between a-b. in the early stage of system design, an expected goal will be planned. For example, given the resource ax, point a is the performance expectation value. In other words, given the fixed resource ax, if TPS can reach point a or even higher, it means that the system performance is up to or better than expected. Through the performance test, we can verify whether the processing capacity of the system has met the expectation
Load test. Located between b-c. Increase concurrent requests to the system until one or more indicators of the system reach the critical value of security, as shown in the figure above. This C is the so-called maximum load. After the request pressure is increased, the processing capacity of the system can not be improved, and the return will decrease. The maximum safe load value of the system can be obtained through the pressure test
Pressure test. Between c-d. In the case of exceeding the safe load, continue to increase the pressure on the system until it reaches the collapse point, i.e. D. in the figure above, the maximum bearing capacity of the system can be obtained through the pressure test
Stability test. Between A-D. At different points of a, B, C and D (representing specific hardware, software and network environment), the system is allowed to run for a long time to detect the stability of the system under different conditions.
It is also recommended to read this book
Monkey testing is conducted by testers, that is, they regard themselves as monkeys and input and operate at will without any knowledge background or understanding of the application.
The goal of the monkey test is to check whether an application or system crashes by providing random input values / data. The monkeys were randomly executed, there were no test cases, and there was no need to understand the full functionality of the system
Mutation testing (or variability testing) is a kind of white box testing, which is the reverse of unit testing.
Generally, the idea of unit testing is to verify whether the code is effective and reliable through test cases, while mutation testing is the reverse. It changes the source code of one of the programs first, and then runs the unit test. If the unit test passes, it may indicate that the test case is ineffective or the test case does not cover the code variation
So mutation testing can in turn validate your test cases and help us identify potential errors that cannot be prevented by current testing
On the contrary, pessimistic test and optimistic route test require the tester to have a “break” attitude, consider all kinds of abnormal situations, and use all kinds of evil, malicious and illegal operations to test the system. Pessimistic tests use incorrect data, invalid data, or input for validation. It verifies that the system can identify anomalies and run as expected.
Non functional testing
Each large organization has an independent team, often referred to as a non functional testing (NFT) team or performance team.
Non functional testing involves testing non functional requirements, such as load testing, stress testing, security, capacity, recovery testing, etc. the goal of NFT testing is to ensure that the response time of software or application meets the business requirements.
For example, it should not take too much time to load any page or system, and it should run well during peak load.
This term is often used interchangeably with “stress” and “load” tests. Performance testing is used to check whether the system meets the performance requirements. It uses different performance and load tools to perform this test.
The scope of performance testing is relatively large. In a broad sense, performance testing includes load testing, stress testing, stability testing, capacity testing and so on. In a narrow sense, performance test refers to whether the test system can achieve the expected value under specific resource conditions, that is, baseline test
Summarize the types of performance tests:
Baseline test: under the given resources, test the best performance and use it as a reference ‘baseline’ for subsequent measurement. Pay attention to the difference between baseline test and benchmark test. In this way, the benchmark is what you want to achieve, such as the 100 sprint world record. The baseline is your score.
Load test: measure the performance of the system under the expected peak production load. Load testing has been outlined above
Endurance test: under the specified load, measure the stability of the system for a long time
Stress test: test system performance under extreme conditions
Recovery testing is used to verify the extent of a crash or disaster recovery in an application or system. It determines whether the system can continue to run after a disaster.
For example, when an application receives data through a network cable, the connection of the network cable is suddenly disconnected. After a period of time, the network cable is plugged in again. The system should start to recover the data lost due to the pulling out of the network cable
After modifying any module or function, testing the application as a whole is called regression testing. The purpose of regression testing is to verify the integrity of software after the original function changes
Some people think that regression testing is regression testing, which means repeating all or part of the same testing work before, which is not unreasonable. Moreover, it is not uncommon that the whole body is triggered by partial modification, which is the purpose of regression testing
Because it is difficult to cover all systems in regression testing, it is usually best to use automated testing tools for these class tests. For example, after modifying the code, run unit test to ensure that other software units are not affected.
In the front-end, the component snapshot testing and some CSS UI tests belong to the regression test type. Their principle is to compare the results generated by the previous test to ensure that there are no unexpected changes
35) risk based testing (RBT)
In risk-based testing, the function or requirement will be tested according to its priority. Risk based testing prioritizes highly critical functions because they have the greatest impact on the business or have a high probability of failure. However, the priority is determined by the business requirements. Therefore, once the priority is set for all functions, the high priority function or test case should be executed first, and then the low priority function. Low priority functions can be tested when there is plenty of time, or not.
Risk based testing should be performed when “there is not enough time to test the entire application, but the software needs to be delivered on time”. It usually needs to be discussed and approved by customers and senior management
- Integrity testing
Integrity testing is used to determine whether a new software version can start formal testing. If a software version should crash at the beginning of use, it indicates that the system is not stable enough and there is no need for further testing. This situation should be called back to the development to avoid wasting time
Take our company as an example:
- List item
In the software design phase, the test team will write smoke test cases for;
- List item
The development team will run its own smoke cases before submitting the version to the test to ensure that there are no major failures;
- List item
After the version is submitted to the test team, the test team will run the integrity test first to check whether there are major bugs that affect the test process, and if so, they will return to development
- List item
If the integrity test is passed, the smoke test will be carried out, and if the smoke test fails, the development will be called back immediately.
- List item
Only after passing the integrity test and smoke test can we enter the formal testing stage.
One of the purposes of this is to reduce the workload of the test team, because they have to connect the test tasks of multiple development teams.
- Security testing
Security is also a huge discipline, and knowledge is updated every day, so security testing is generally carried out by special security teams, who use various hacker means to conduct penetration test on the system.
The purpose of security testing is to ensure that the application or website is free from internal and external threats. This test includes preventing malicious programs, viruses, and verifying the security of authorization and authentication processes.
It also checks how the software reacts to any hacker attacks and malicious programs, and how to maintain the software to protect data security after being attacked by hackers
Smoke test: every time the development team submits a new build, the software test team will verify the build first and ensure that there are no major problems. If there are major problems, they will be directly called back to the development team
How to understand smoke test in general? This belongs to the hardware industry. After a hardware or hardware component is changed or repaired, the device is powered on directly. If there is no smoke, the component passes the test. For example, if you power up the Samsung Note7, if it doesn’t explode, it means that it has passed the “smoke test” (it’s not easy to be a mobile phone tester, it’s easy to be life-threatening)?
The test team will not further perform detailed tests until the build is stable. The smoke check checks for a stop defect in the build, which prevents the test team from further detailed testing. That is, if testers find that the main functionality doesn’t work, they reject the build and return it to the development team.
Smoke testing is usually performed prior to regression testing or other detailed testing
Static testing is a bit like code review, which is executed without executing any code (that is, without running the application). It involves inspection, review, and walkthrough. For example, checking code syntax, naming conventions, and project organization.
Static testing is not only applicable to code, but also to test cases, test plans and design documents. If defects are found in the static testing phase, the cost of defects can be minimized. For example, it is better to find problems in the design phase than in the development phase or even in the production environment
For example, static testing may include:
Use lint tools to check the specification of programs. The related tools include eslint, tslint, stylint, etc., and even typescript can be included in this category
Code review. There are some problems that cannot be covered by lint tools, such as code logic, exception capture, project organization, memory leak, etc., which need to be reviewed manually
Checking whether the code is consistent with the design, whether it meets the software requirements, outline and detailed design can not only identify the code problems, but also find out whether the requirements or design are correct earlier.
- Stress testing
Through the stress test, it simulates the failure mode and time of the system under the pressure beyond its specification, and finds out the collapse point of the system. This test is executed under high load conditions, such as accessing data exceeding the capacity limit, executing complex database query, continuously and violently inputting to the system or loading into the database.
- System testing
System testing is carried out on a complete integrated system, that is to say, the system test is generally conducted after the integration test. After the integration test, the system becomes a whole. On this basis, the system test verifies whether the system meets the business requirements in the real running environment. This is a black box test, based on the overall requirements specification, covering all the composite parts of the system.
In fact, a specific system is not a testing stage. At this stage, there will be many kinds of tests, and the work of the test team of the general company is focused on this one. It generally includes:
Function test: as mentioned above, test whether the system meets the business requirements as a whole
Various non functional tests: such as recovery test, performance test, stress test, safety test, etc.
Summarize the purpose of system testing
Ensure that the application as a whole works well
Ensure applications meet business requirements
Ensure that the application can run well in the real environment. For example, do some non functional tests to verify the robustness of the system
In fact, system testing is very similar to the end-to-end testing mentioned above. They require the system to be tested as a whole. You can simply expand the comparison
- Unit testing
Testing independent software units or modules is called unit testing. It is usually done by developers, not by testers, because it requires a detailed understanding of the internal programming and code.
Unit testing is the type of test most closely related to our developers. Its test object is software unit. A software unit can be a function / method, a class, or a GUI component, etc.
This is a white box test, so it needs to be done by the developer himself, because only the developer knows the internal implementation of the unit. Unit testing generally uses test coverage to verify the completion of unit testing
The common front-end unit testing tools include jet, mocha, jasmine, etc. the following is a typical BDD style unit test organization:
43) usability testing
Usability testing is used to test the user friendliness of an application. It will verify that the new user can easily understand the application process. If the user is in trouble, the tester should record and provide help. It can be considered that the usability test is to check the navigation of the system
Vulnerability testing involves identifying vulnerabilities in software, hardware and networks. If the vulnerability is vulnerable to attack, or vulnerable to viruses and worms, hackers or malicious programs can take control of the system.
Therefore, it is necessary to check these systems for vulnerabilities before putting them into production environment.
Capacity testing is a non functional test performed by a performance testing team. The capacity test checks the system behavior and response time when the application encounters large amounts of data. This large amount of data may affect the performance of the system and the speed of processing time.
White box testing
White box testing, also known as glass box testing, structural testing, logic driven testing or code based testing, is based on the internal logic of application code. That is, testers should know how the internal software and code work, and test all the logical paths. The unit tests and static tests mentioned above are typical white box tests. Basically, white box tests can be equivalent to unit tests
Logical path includes statement coverage, decision coverage, conditional coverage, decision / condition coverage, conditional combination coverage and path coverage
The software test types mentioned above are only a part of the test. There are more than 100 test types in fact. (718897738) welcome to exchange software testing together. The group provides the latest free video + tools to download. But not all test types will be used by all projects, so I will just list some common software test types.
In addition, different organizations may have different definitions or processes, but the basic concepts are the same everywhere. As the project, requirements, and scope change, these test types, processes, and implementation methods evolve.