How to implement continuous testing in Devops


In the past decade, the demand for software development has grown dramatically. Software has become a key competitive advantage for companies, especially if your company is in the SaaS category. By implementing traditional processes such as waterfall in SDLC, organizations are now transitioning to agile to deliver software on the market at a faster rate. In response to rad (rapid application development), there are many new methods, such as CI / CD, Devops, shift left key testing, in order to better build, develop and optimize software delivery.
Even so, trying to maintain quality and speed at the same time is a real challenge, and test methods can help or reduce the whole acceleration process. Today, we will explore the importance of continuous testing in Devops. In this article, what is continuous testing? It will also help eliminate misconceptions about continuous testing. We will also explore the challenges involved in continuous testing in Devops, as well as best practices, to help you perform the continuous testing process as a professional.

What is continuous testing?

Continuous testing is an end-to-end quality maintenance process in which the team continuously conducts various automated tests. At the same time, it analyzes various business risks related to the latest software development and provides rapid feedback for developers. This feedback helps to identify defects and errors at an early stage and encourages developers to optimize their code in subsequent phases of the SDLC (software development lifecycle).

Unlike traditional testing methods that deliver results at the end of the development cycle, continuous testing takes place in multiple phases, including development, integration, transition, and production environments. Continuous testing ensures that defects and problems are resolved early in the development process, improving overall quality and saving a lot of time and money.

Benefits of continuous testing

Conventional testing techniques are largely dependent on manual testing and automated testing that requires periodic updates, which can slow the delivery process. This is what modern methodologies like agile, Devops, continuous integration, and continuous delivery explore.

Continuous testing in Devops can be achieved by:

  • Continuous risk analysis: there may be a version (candidate release) that passes all available tests, but is not ready to be released by the business head, and the continuous test will assess these risks at each stage.
  • Keep in mind the user experience: continuous testing is a process that can easily adapt to changing customer requirements. By constantly updating your application based on customer feedback, integrating continuous testing with Devops can help make your software more robust and stable. From the customer’s point of view, you have the flexibility to write effective test cases. Proper testing of the user experience is critical to evaluating the end-user experience of all relevant technologies in the front-end and back-end.
  • Help security: continuous testing establishes a support system that ensures the security of applications against unexpected changes and attacks that may be encountered after deployment. In the accelerated development process, even in the case of software failure, it can ensure that the system is stable and recoverable.
  • Continuous integration from the beginning: continuous testing expects tests to be embedded early in the development process, rather than being processed before release. Testing is continuously integrated into the software delivery pipeline and Devops tool chain.
  • Covering functional and non functional tests: continuous tests can simulate all types of functional tests, such as cross browser tests, regression tests, integration tests, API tests, unit tests; and non functional tests, such as availability tests, security tests, reliability tests, scalability tests, etc.
  • Timely feedback without any bottlenecks: continuous testing evaluates every layer of modern architecture at the right stage of the delivery pipeline and provides actionable feedback at the right stage of the delivery pipeline without creating lengthy queues.
  • Save time, money and resources: early detection of bugs can not only save resources for publishing windows, but also help you save a lot of money and resources. Continuous testing reduces the time and resources needed to find and fix defects by using defect prevention strategies such as development testing or move left testing.

Misconceptions about continuous testing in Devops

  • “Will lead to tester unemployment”: testers have an idea of the framework: they can see how customers interact with it. Despite the rapid development of automation, it has not completely replaced manual testing. The whole purpose of introducing automation is to provide more time for manual testers to propose more effective test plans. There are still important manual activities, such as exploratory testing and usability testing, and testers should continue to perform them, which is a feature of their work. Of course, they need to acquire effective skills as software testers.
  • “Only testers can contribute to continuous testing”: some or all parts of continuous testing are critical to any type of team and its team members. For example, having testers work with developers might enable them to create advanced automated test suites. In this way, developers have the opportunity to understand the tester’s point of view, and testers can also learn and improve their test automation skills.
  • “Continuous testing means continuous execution of test cases”: as I’ve explained, continuous testing has many other functions. It not only performs functional and non functional testing, but also continuous testing covers moving from left (unit, component, coverage, comprehensive risk assessment) to “right” (monitoring / APM, in production testing, customer experience, virtualization testing).
  • “Continuous testing and automated testing are the same”: through automation, enterprises are trying to implement agile testing strategy: “test early, test frequently, test everywhere”. The lengthy test script will be executed automatically and updated regularly. With the help of automatic test tools, automatic test can be used to execute continuous test, thus reducing the human error. However, keep in mind that automated testing and test automation are slightly different. Through test automation, ensure the speed and efficiency of the whole delivery process through the pipeline. Automated testing can be automated, traditional testing methods can automatically manage and track the whole process of different tests. Continuous testing includes automated testing and automated testing to maintain quality and speed. Automated tests are subsets of continuous tests and should not be confused.

Challenges of continuous testing in Devops

  • A huge one-off investment: building a test environment and an automation framework requires a lot of expertise and effort. The biggest difficulty in obtaining the scope of test automation is the time and cost associated with establishing an effective automation framework. In order to realize the test automation project, JIRA, asana and other comprehensive test management platforms simplify the process.
  • Testing a wide range of complex architectures: modern applications are widely distributed, the adoption of agile and parallel development processes is increasing, and end-to-end functional testing usually requires access to third-party services or mainframe, which can only be used for limited capacity or inconvenient time. You can solve this problem by using service virtualization to simulate aut (application under test) interactions with missing or no dependencies. It can also be used to ensure that data, performance, and behavior are consistent across test runs.
  • Non scalable test suites: another reason teams avoid continuous testing is that their infrastructure is not scalable enough to run test suites continuously. This problem can be solved by focusing tests on company priorities, splitting the test base, and parallelizing tests with application release automation tools.
  • Lack of coordination between teams: finding the right skilled automation experts is also a challenge. Continuous testing in Devops requires a high degree of coordination among product managers, developers, and testers. Coordination is the area most companies struggle with. Since ancient times, there has been a cultural disconnect between them. The quality inspection team is isolated and can be solved through appropriate employee engagement strategies and awareness.

Main instructions for successful continuous testing in Devops

  • Create a powerful user story: continuous testing means quality and granularity testing from the beginning. Ensure good business requirements are obtained to start development. To ensure that user stories are testable and have good acceptance criteria, manual testing with a more exploratory attitude may help achieve good results.
  • Collaboration: from a cultural point of view, continuous testing in Devops is successful if everyone demonstrates the quality and cooperation of the team. Describe test cases before you start coding or writing tests as needed. In any case, developers and test automation architects should work together to ensure that the code used for test automation is optimized. Teams can also use tools like slack to collaborate on test results to speed feedback and debugging.
  • Keep it simple and logical: reduce unnecessary test objects, such as extensive test plans and test cases, and reduce test waiting time. The test should be consistent, incremental and repeatable; the results should be quantifiable and meaningful.
  • Testing everywhere: testing must be done at all stages of the delivery pipeline, covering all aspects of the entire environment, whether it’s the production or QA environment. By testing at each stage and providing continuous feedback to developers, you can help improve the quality of software development.
  • Automated testing: automated testing plays an important role in the successful implementation of continuous testing in Devops. It is important to adhere to the test automation pyramid and focus on automating test scripts to achieve the latest updates in web applications. You can’t achieve 100% automation, but the more you can automate the process, the faster you can perform continuous tests.
  • Embrace CI / CD (continuous integration / continuous delivery): developers should adopt continuous integration – by integrating code into shared repositories (such as bitbucket and GitHub) multiple times a day. When automated tests are implemented using CI servers, each build immediately begins continuous testing. Warning, whether the test results pass or not, they can be sent directly to the development team in real time. With regular integration, you can detect and locate errors more easily and quickly. Once all tests have been completed, updates can be delivered continuously to production without hesitation.
  • Choose GUI Based API: Devops and agile teams work with short release cycle, fast feedback cycle and frequent changes, which makes it difficult to maintain GUI tests. GUI testing takes longer to provide feedback and requires a lot of rework. For modern applications with multi tier architecture, it is important to verify the back-end services and functional paths. API testing is more stable, and it is recommended to use the same. GUI testing is limited to system testing, mobile testing and black box testing; API testing involves many practices, such as unit testing, function regression testing, load testing, security testing, web interoperability testing, etc.
  • Thinking beyond automation: just like human brain work, the implementation of artificial intelligence (AI) and the design of intelligent machines (with the help of IOT, quantum computing, robots, etc.), heuristic thinking on each stage of delivery pipeline can accelerate the development of the whole process. SDLC.


In order to move quickly and achieve faster results, we have to make sure that we make the right products from the beginning. Modifying products for bug fixes has never been easy because the cost of bug fixes later found in the pipeline is high. Instead, you must test in the right way, using synchronous delivery processes (CI / CD, Devops), test methods (API testing, service virtualization), a stable test platform, and functional and non functional aspects of automated testing.

By integrating the traditional disconnect between teams, testers and developers can learn and execute successful automation scripts with appropriate expertise, and easily optimize the software architecture.

Continuous testing in Devops is the primary (not the only) method of continuous quality. This is a step towards higher quality products through continuous delivery.

Once continuous testing is implemented in Devops, an ideal opportunity arises to consider different approaches that do not include running tests to identify defects or problems, but rather to prevent coding. In this way, developers are encouraged to build the best flawless product from the beginning.

  • Solemnly declare: the article forbids the third party (except Tencent cloud) to reprint and publish. It’s a test of the original cause of the matter. Copy seven of my original articles on the home page and pull black. Won’t your conscience hurt?

Selected technical articles

  • Java one line code printing heart
  • Chinese version of Linux performance monitoring software netdata
  • Interface test code coverage (jacoco) scheme sharing
  • Performance Testing Framework version 2
  • How to test the performance of Linux command line interface happily
  • Diagram HTTP brain map
  • Automatically turn a swagger document into test code
  • Five line code to build static blog
  • Research on the testing framework of linear interface based on Java

Selected non-technical articles

  • Why choose software testing as a career path?
  • Programming thinking for all
  • 7 steps to become an excellent automation test engineer
  • 7 skills to become an automated test
  • Automated test life cycle
  • How to introduce automated testing in Devops
  • Summary of web automation test failure reasons
  • How to introduce automated testing in Devops
  • How testers become agents of change
  • Tips for writing test cases

Recommended Today

Review of SQL Sever basic command

catalogue preface Installation of virtual machine Commands and operations Basic command syntax Case sensitive SQL keyword and function name Column and Index Names alias Too long to see? Space Database connection Connection of SSMS Connection of command line Database operation establish delete constraint integrity constraint Common constraints NOT NULL UNIQUE PRIMARY KEY FOREIGN KEY DEFAULT […]