Coding Devops series lesson 1: building continuous delivery platform based on open source tool chain


Current software development trend

Several popular technologies in the development of IT industry today are microservices, which split an original system into multiple systems, which means that multiple systems need to be built, tested, deployed and operated.

The second is the agile development mode, which requires a unit that can be deployed independently for rapid development, rapid testing, and rapid deployment online to achieve rapid iteration.

With the rapid development of container technology, more and more applications migrate to containers.


At this time, there will be some problems. If the current software delivery continues to use the traditional mode, it will cost a lot of manpower and material resources, and there are a lot of repeated deployment tasks, and the delivery can not be fast. So, is there a better delivery method to meet the current software development trend? The answer must be yes. It is in this context that Devops was born.

Introduction and characteristics of Devops

Devops is a combination of development and operations, that is, the integration of development, operation and maintenance. Generally speaking, it is to realize software delivery automation as much as possible and guarantee the quality of software delivery through a series of tools and some specifications.

Generally speaking, Devops has the following characteristics: first, automation. Through the introduction of a series of tools, the whole delivery project automation from demand to online is realized, and the necessary links are confirmed manually.

The second point is standardization. Tools alone are not enough. A series of specifications are needed, such as version management specification, test management, test data management, etc.

The third point is to shorten the lead time. The delivery process is basically one button or fully automatic, which saves unnecessary links and shortens the delivery cycle.

The fourth point is that the delivery quality is guaranteed. During the delivery process, static code scanning, unit testing, interface testing and other links can be introduced, and the quality threshold is set for each link, so as to reduce the probability of production bugs, and truly prevent problems in the bud.

Based on the above four characteristics, our whole product has been improved correspondingly. The automation of delivery process liberates the labor force and ensures the delivery quality, which naturally brings greater benefits.


Tool set selection

Version control system

Version control system (VCS) is also called source code management system. As the name implies, it provides the most basic version control function. It will keep the modification history in the process of file modification, so that users can easily view the modification history of the file. And it is convenient for users to undo the modification of the file.

At present, there are two version control systems widely used in the industry. The first is SVN, which is an open source version control system. It is developed based on CVs. It is used for multiple people to jointly develop the same project and share resources. It is simple, fast and easy to operate, but it can not achieve branch management, so it depends on the network.

The second is git, which is a free and open source distributed version control system, which is used to deal with any large or small projects quickly and efficiently. As an open source distributed version control system, it can effectively and efficiently handle various project version management and achieve good branch management.


Continuous integration

Continuous integration also introduces a common tool Jenkins. I believe many partners have used Jenkins. It is an open source automation server. As an extensible automation server, Jenkins can be used as a simple CL server or become the continuous delivery center of any project. Its community is very active, plug-ins are relatively complete, can integrate requirements management, version management system, etc.


Continuous testing

For continuous testing, we will have a test layer, which is divided into unit test, interface test and joint debugging test. Unit testing is a method level test. Developers need to write test classes for the source code. The purpose is to verify whether there is a problem with the functional code. You can use JUnit + jmock to implement and connect to the pipeline.

Interface testing is to test the exposed interface or the method of providing services for the system. It is necessary to write test cases, and JMeter can be integrated into Jenkins to realize interface automatic test and result collection.

The joint debugging test refers to the connectivity test between systems, which needs manual intervention.


Continuous deployment

What we deploy involves not only the application deployment package, but also some configuration files and database scripts. In most cases, configuration files are bound to the environment. Each set of environment uses a set of configuration files, which requires unified management of configuration files. When publishing, select corresponding files or replace them dynamically.

Database scripts need to incorporate the SQL change files into the version management system, and execute the change SQL incrementally during the release.

Continuous integration pushes the build package to the product library and manages it according to certain specifications. During deployment, the corresponding version of application package deployment is pulled from the product library. Applications may be deployed to physical machines, virtual machines, private clouds, or public clouds.


Branch strategy scheme

In the whole delivery process, version control is the most important part, and version control is related to branch strategy. If the early branching strategy is not done well, the later version control must be very bad. Only when the front is done well, can it be done well in the back.

The necessity of branch management

I believe that all partners will have a certain understanding here. Branch management can help us avoid code loss. If there is no reasonable branch management and a certain function has not been developed, code push to remote will affect other functions. If not pushed to remote warehouse, local data loss will lead to code loss.

At the same time, branch management can also improve code quality. After setting branch permissions, the merging of feature branches to trunk branches requires the review of master role to ensure the code quality of main branches.

Finally, with the software iteration, the version number also changes. In order to trace the requirements, change sets and online bug modifications of each version, it is necessary to design reasonable branching strategies and manage them to requirements and deployment packages.


Characteristics of reasonable branching strategy

First of all, a reasonable branching strategy must conform to the business scenario. There is no best, only appropriate. It is the most important to design a branch strategy that is in line with your own business scenarios.

Second, the hierarchical structure of branches is clear. A clear branch hierarchy can better reflect the function of each branch.

Third, avoid formalism. We should not design multi-layer branches for branch management, which results in multiple merges for one submission and loses the effect of efficient payment.

The last is the permission control, which needs to set the corresponding permissions for the main branches. Only the designated personnel can have the permission combination to improve the code quality.


Introduction to pipeline

All delivery processes are based on pipeline. Pipeline is commonly known as pipeline, which is also known as job in Jenkins. Multiple building units form a pipeline, such as code compilation, unit test and code scanning. There are two ways to choreograph a pipeline: configuring the implementation in Jenkins interface and writing Jenkins file to realize the arrangement of links.

We mainly recommend using Jenkins file for choreography, because in the current business scenario, our proposal is “all configurations are code”, which can ensure that all our configurations are coded. Even if our Jenkins server is down or deleted, our Jenkins file can be stored in the source code management system.

Jenkins file is a script file recognized by Jenkins. All construction steps are written to the file in the form of code according to certain syntax. Creating pipeline is to specify the file path.


In the full video, we will continue to bring you the branching strategies in several scenarios, as well as the design, measurement and feedback of pipeline. Click to see the full video.