Small hub reading:
There are so many script codes. If it’s not operation and maintenance, you may not be able to write it. Ha ha. It’s good for ordinary people to use Jenkins in full, but it’s troublesome when there are too many codes. Moreover, Jenkins should be used with scripts, which is more in line with the business.
Author: catalpaflat
https://juejin.im/post/5cf0ed…
Application deployment is a process that must be faced with in development, testing and online. Especially with the emergence of microservice architecture, operation and maintenance deployment is gradually separated from the single deployment, and becomes more and more complex.
However, apart from multi language, multi environment, cluster and distributed deployment. Just talk about incremental deployment and full deployment
1. Incremental and full deployment
Deployment: in addition to the initial deployment of the project, the ideal situation is as follows:What is changed in the new version will be updated
1.1 incremental deployment
1.1.1 introduction to incremental deployment
Incremental deployment generally refers to extracting the increment (including code, executable file or configuration) between the current version and the version to be deployed in each deployment process, and only the incremental part is updated during the deployment process.
1.1.2 common deployment process
- Use code management tools (SVN, GIT, etc.) to extract the increment between the two versions, and combine the incremental changes in other aspects.
- According to the incremental part, the specific deployment method is formulated, the deployment script is written, and the incremental deployment package (including confused code, etc.) is prepared.
- Distribute and deploy the incremental deployment package to the target environment that has run the previous version, and complete the version upgrade of the system.
1.1.3 advantages of incremental deployment
- Fast deployment. Only the incremental part is updated each time to shorten the deployment time
- Reduce the amount of change. To reduce the change range of the whole system, some configuration contents do not need to be updated every time
- Improve security. Due to the increment of each detachment, it can avoid the leakage of all code
1.1.4 disadvantages of incremental deployment
- Incremental deployment will reduce deployment efficiency if there are other external deployment environment dependencies
Incremental deployment is not like
- When there are many deployment environments, the requirement for repeatability is high
- Incremental deployment becomes unfriendly to rollback operations
1.2 how to choose increment or total quantity
Most of the existing automated deployment are full deployment, but full deployment also has some disadvantages. However, there are some strategies for filtering:
- All configurations and materials (deployment packages, external configuration files, etc.) deployed in advance can improve efficiency and speed
- Using gray publishing or load balancing methods to reduce the impact of full deployment on application availability
For most state independent deployment units (applications, modules, microservices, etc.) in modern systems,In general, full deployment should be the best choice。 The state dependent deployment unit (database, etc.) is still suitable for incremental deployment logic.
2. Enter the theme
Some of the incremental and full deployment scenarios were described earlier. Next, we will talk about how to use shell script and git log for incremental deployment
2.1 premise environment
- Java project
- Maven
- Git as code repository
2.2 shell script
Shell novice, not perfect writing, light spray.
2.2.1 module of the whole shell script
- Git environment preparation
- Maven compiles the project to be built
- Create incremental deployment folder
- Retrieve project target directory
- Git diff is used to retrieve the difference between two commit, and then the corresponding file is copied to the “incremental folder”
2.2.2 git environment preparation
#Git environment
if [[ ! -d ".git" ]]; then
ECHO error: please init Git Repository
exit 1;
fi
if [[ ! -z ${branch} ]]; then
git checkout ${branch}
fi
#Get default commit hash
if [[ -z "$begin_hash" ]] && [[ -z "$end_hash" ]] ; then
for p in $(git log --pretty=oneline -2) ; do
if [[ ${#p} -eq 40 ]]; then
if [[ -z ${begin_hash} ]]; then
begin_hash=${p}
else
end_hash=${p}
break
fi
fi
done
fi
is_begin_has=false
#Is it the latest commit
if [[ $(git log --pretty=oneline -1) == *${begin_hash}* ]]; then
is_begin_has=true
fi
#If it is not the latest branch commit, it will be rolled back to the original version. At that time, the original Maven configuration does not support compile or there may be build failure (such as using local warehouse / private warehouse, etc.)
if [[ ${is_begin_has} = false ]]; then
project_path=$(pwd)
project_name=${project_path##*/}
cd ..
build_project_name=${project_name}_build_temp_project
if [[ ! -d ${build_project_name} ]]; then
mkdir ${build_project_name}
fi
\cp -rf ${project_name}/. ${build_project_name}
cd ${build_project_name}
git reset --hard ${begin_hash}
fi
Copy code2.2.2.1 check whether git warehouse code is used
if [[ ! -d ".git" ]]; then
ECHO error: please init Git Repository
exit 1;
fi
Copy code2.2.2.2 check whether it is necessary to switch branches
if [[ ! -z ${branch} ]]; then
git checkout ${branch}
fi2.2.2.3 do you need to set the commit value of the default build
If you do not add — begin_ Hash = and — end_ Hash = to assign a value, the latest two commit are used for incremental deployment by default.
Get the hash of the last two commit through git log — pretty = oneline – 2
#Get default commit hash
if [[ -z "$begin_hash" ]] && [[ -z "$end_hash" ]] ; then
for p in $(git log --pretty=oneline -2) ; do
if [[ ${#p} -eq 40 ]]; then
if [[ -z ${begin_hash} ]]; then
begin_hash=${p}
else
end_hash=${p}
break
fi
fi
done
fi
Copy code2.2.2.4 verify the begin of parameter transfer_ Whether the hash value is the latest commit hash of the current branch
If it is not the latest commit hash of the current branch, you need to roll back to the corresponding commit to build and compile the project
if [[ $(git log --pretty=oneline -1) == *${begin_hash}* ]]; then
is_begin_has=true
fi
Copy code2.2.2.5 if begin_ Hash is not the latest commit hash
Begin_ The value of hash is not the current latest commit hash. You need to roll back to the corresponding commit for build compilation.
- Copy the existing project to a new directory environment
- Reset the project to the new directory environment to build the project
if [[ ${is_begin_has} = false ]]; then
project_path=$(pwd)
project_name=${project_path##*/}
cd ..
build_project_name=${project_name}_build_temp_project
if [[ ! -d ${build_project_name} ]]; then
mkdir ${build_project_name}
fi
\cp -rf ${project_name}/. ${build_project_name}
cd ${build_project_name}
git reset --hard ${begin_hash}
fi
Copy code2.2.3 Maven compiles the project to be built
Compile the project to generate the corresponding class file and related configuration file
mvn clean compile -q -DskipTest
Copy codeIf the local warehouse is used in the historical version but not configured in maven, it can be reconfigured and introduced through scope and SystemPath, such as:
<dependency>
<groupId>cn.catalpaflat</groupId>
<artifactId>core</artifactId>
<version>1.0.0</version>
<scope>system</scope>
<systemPath>${project.basedir}/lib/core-1.0.jar</systemPath>
</dependency>
Copy code2.2.4 create incremental deployment folder
In order to prevent the incremental folder from being deleted or committed to git repository, it can be unified into a directory and ignored by. Gitignore. You can compare the differences between each incremental deployment
build_path=build-path/
current_date=`date +%Y%m%d%H%m%s`
if [[ ! -d "$build_path$current_date" ]]; then
mkdir -p ${build_path}${current_date}
else
rm -rf ${build_path}${current_date}
mkdir -p ${build_path}${current_date}
fi
Copy code2.2.5 search project target directory
If the project is a maven project and it is a java project, due to the Maven multi module situation, it is necessary to retrieve the compiled code path under each module for subsequent copy of class and other files.
default_target_paths=()
default_java_file=java
module_index=0
#Retrieve whether the current project is Maven multi module development, retrieve recursively, and set the compiled code location (only Java type is provided temporarily)
obtain_module(){
for module in ` cat ./pom.xml | grep '<module>' | awk -F '>' '{print $2}' | awk -F '<' '{print $1}' `
do
cd ${module}
if [[ ! -d "/pom.xml" ]]; then
module_exist=`cat ./pom.xml | grep '<module>' | awk -F '>' '{print $2}' | awk -F '<' '{print $1}'`
if [[ -z ${module_exist} ]]; then
if [[ ! -d "/target" ]]; then
if [[ -z $1 ]]; then
default_target_paths[module_index]=${module}/target/classes
else
default_target_paths[module_index]=$1/${module}/target/classes
fi
((module_index++))
fi
else
if [[ -z $1 ]]; then
obtain_module ${module}
else
obtain_module $1/${module}
fi
fi
fi
cd ..
done
}
obtain_module
Copy code2.2.6 retrieve and copy change files to incremental folder
- Git diff — name only check the file differences between two commit
- And start_ Hash commit compiled code is copied to the increment folder for subsequent packaging and deployment
#Git diff -- name only is used to realize the file difference between two commit, and start_ After the hash code is compiled, copy the difference files to the "incremental folder" for subsequent incremental deployment
for file_path in $(git diff --name-only ${begin_hash} ${end_hash}) ; do
package_path=${file_path%/*}
file_name=${file_path##*/}
file_type=${file_name##*.}
#Check whether the file folder is created
if [[ ${package_path} != *.* ]]; then
if [[ ! -d "./${build_path}${current_date}/$package_path" ]] ; then
mkdir -p ./${build_path}${current_date}/${package_path}
fi
fi
#Java or not
if [[ ${file_type} = ${default_java_file} ]]; then
module_path=${package_path##*java}
file_class_name=${file_name%.*}
module_type=${package_path%%/*}
#Check which Maven module path is in
for default_target_path in ${default_target_paths[@]}; do
target_module_path=$(echo ${default_target_path} | awk -F '/target/' '{print $1}')
file_target_module_path=$(echo ${package_path} | awk -F '/src/' '{print $1}')
file_target_package_path=$(echo ${package_path} | awk -F '/src/main/java/' '{print $2}')
default_module_type=${default_target_path%%/*}
if [[ ${target_module_path} = ${file_target_module_path} ]]; then
#Check the target directory of the corresponding Maven module and perform CP operation
cp -afx ${default_target_path}/${file_target_package_path}/${file_class_name}* ./${build_path}${current_date}/${package_path}
fi
done
else
#Non java files, directly copy the files to the corresponding directory
if [[ ${package_path} != *.* ]]; then
if [[ ! -d "./${build_path}${current_date}/$package_path" ]] ; then
mkdir -p ./${build_path}${current_date}/${package_path}
fi
else
package_path=${package_path%/*}
fi
cp -afx ${file_path} ./${build_path}${current_date}/${package_path}
fi
doneSource code through train!!! : https://github.com/CatalpaFla…
So far, the preliminary version 1.0 can be used
(end)
Recommended reading:
Great, spring boot + Vue front and back separation complete introductory tutorial!
Share a set of springboot development blog system source code, as well as the complete development document! Speed save!
GitHub’s top 100 Java open source projects, covering a variety of technology stacks!
The latest interview questions and answers in 2020