Official next generation docker image construction artifact — buildkit

Time:2021-1-22

Official next generation docker image construction artifact -- buildkit

Buildkit is the next generation image building artifact launched by docker official community, which can build docker image more quickly, effectively and safely. Docker v18.06 has integrated this component. Buildkit can be used for a variety of export formats (such as OCI or docker) and front-end support (dockerfile), and provides efficient caching and parallel build operations. Buildkit only needs container runtime to execute. Currently, the supported runtime includes containerd and runc.

Optimization of construction steps

One of the most frustrating problems with the original build provided by docker is the order in which the dockerfile instruction executes the build steps. After the introduction of multi-stage build, the build steps can be grouped into separate logical build tasks in the same docker file.

Sometimes, these build phases are completely independent of each other, which means they can be executed in parallel – or not at all. Unfortunately, the traditional docker image construction can not meet this flexibility. This means that the build time is usually longer than absolutely necessary.

In contrast, buildkit creates a dependency graph between the build steps and uses it to determine which elements of the build can be ignored; elements that can be executed in parallel; and elements that need to be executed in sequence. This can perform builds more efficiently, which is valuable to developers because they can iterate over mirror builds of their applications.

Efficient and flexible cache

Although the cache construction steps are very useful in the old docker image construction, the efficiency is not as good as expected. As a rewriting of the build back end, buildkit improves on this aspect and provides a faster and more accurate caching mechanism. Use the dependency graph generated for the build, and based on the instruction definition and build step content.

Another great benefit of buildkit is that it comes in the form of build cache imports and exports, just as kaniko and makisu allow the build cache to be pushed to the remote registry, so does buildkit, but buildkit gives you the flexibility to embed the cache into the internal registry. Mirror (inline) and put them together (though not supported by every Registry), or import them separately. You can also export the cache to a local directory for later use.

The ability to import a build cache plays its role when building a build environment from scratch without any previous build history: importing a “warm-up” cache is particularly useful for temporary CI / CD environments.

workpiece

When using the old docker image builder to build an image, the generated image is added to the cache of the local image managed by the docker daemon. Need separatedocker pushUpload the image to the remote container image registry. The new artifact building tool enhances the experience by allowing you to specify the image push when building calls. Buildkit is no exception. It also allows you to output images in several different formats: files in the local directory, local tarball, a local OCI image tarball, a docker image tarball, a docker image stored in the local cache, and a docker image pushed to the registry R image, there are many formats!

Extended grammar

For the docker build experience, one of the many repeated function requests is to safely handle the confidential information needed in the image build process. Moby project has resisted this requirement for many years, but with the help of the flexible “front-end” definition of buildkit, it provides an experimental front-end for buildkit, which extends the dockerfile syntax. The extended syntax provides a useful complement to the run dockerfile instruction, including security functions.

RUN --mount=type=secret,id=top-secret-passwd my_command

The dockerfile of the experimental front end can be used to temporarily mount the secret key for the run instruction. use--secretFlag provides the secret key to the build for docker build. The SSH mount type can be used to forward SSH proxy connections for secure SSH authentication.

Application scenarios of buildkit

There are many other features of buildkit that can greatly improve the skills of building container images. If it’s a common tool for many different environments, how to use it?

Depending on your work environment, the answers to this question are varied. Let’s see.

Docker

Although buildkit is not the default build tool for docker, it can be considered as the preferred build tool for docker (v18.09 +). Of course, it is not supported on Windows platform at present.

The temporary solution is to set environment variablesDOCKER_BUILDKIT=1

If it is to take effect permanently, it will"features":{"buildkit": true}Add to the configuration file of the docker daemon.

In this configuration, due to the current limitations in the docker daemons, docker does not fully display all the functions of buildkit. Therefore, docker client cli has been extended to provide plug-in framework, which allows plug-in extension and provides available cli functions. One is calledBuildxThe experimental plug-in of will bypass the old build functions in the daemons and use the back-end of buildkit for all builds. It provides all familiar mirror build commands and functions, but it is expanded by some additional functions specific to buildkit.

Both buildkit and buildx support multiple builder instances, which is an important function. In fact, it means that a builder instance field can be shared for building; maybe a project is assigned a set of builder instances.

$ docker buildx ls  
NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS  
default * docker  
  default default running linux/amd64, linux/386

By default, the buildx plug-in targets at the docker driver, which uses the buildkit library provided by the docker daemons and has its inherent limitations. Another driver is docker container, which can launch buildkit transparently inside the container to perform the build. Function cli provided in buildkit: whether this is an ideal workflow depends entirely on the choice of individuals or companies.

Kubernetes

More and more organizations put build into kubernetes, and usually build container image as a part of CI / CD workflow in pod. When running the buildkit instance in kubernetes, there is a problem. Each deployment strategy has its own advantages and disadvantages, and each strategy is suitable for different purposes.

Official next generation docker image construction artifact -- buildkit

In addition to using docker cli to start developer oriented builds for buildkit, builds can also be triggered by a variety of CI / CD tools. Container image building using buildkit can be performed as Tekton pipeline task.

conclusion

This paper mainly talks about many features and use scenarios of buildkit.

At present, there are many similar tools, such as RedHat’s buildah, Google’s kaniko or docker’s buildkit.

However, buildkit is officially provided, which is better combined with docker itself.

Recommended Today

Practice of query operation of database table (Experiment 3)

Following the previous two experiments, this experiment is to master the use of select statements for various query operations: single table query, multi table connection and query, nested query, set query, to consolidate the database query operation.Now follow Xiaobian to practice together!Based on the data table (student, course, SC, teacher, TC) created and inserted in […]