Build a safe, reliable and minimized docker image

Time:2020-6-29

Build a safe, reliable and minimized docker image
Containerized deployment is more and more used in enterprise production environment. How to build itReliable, safe, minimizedOfDockerImage is becoming more and more important. This article will be aimed at this problem, through the way of principle and practice, to help you roll over from head to toe. The article is relatively long, mainly through five parts to explain the container image. namely:

  • Image construction
    Explains the process of manual and automatic image construction.
  • Image storage and unionfs joint file system
    This paper introduces the hierarchical structure of image, the union file system of unionfs, and the implementation of image layer on unionfs.
  • Minimize mirror construction
    Explains why you need to minimize mirrors and how to minimize builds.
  • Reinforcement of container image
    The concrete way of container image reinforcement.
  • Review of container image
    Container images in high-quality projects also need to be reviewed like code.

Readers can read selectively according to their own situation.

The original is sent to my personal website: GitDiG.com Refer to the link to build a safe, reliable and minimized docker image

1. Build an image

1.1 manual build

Manual buildDockerThe flow chart of the image is as follows:

Build a safe, reliable and minimized docker image

Now follow the process in turn to manually build a simpleDockerMirror image.

1.1.1 create container and add file

takebusyboxAs the basic mirror image of this experiment, it is small enough and large enough1.21MB


$: docker run -it busybox:latest sh
/ # touch /newfile
/ # exit

Through the above operations, we have completed the first three steps of the flow chart. A new container is created and a new problem is created on it. It’s just that after we exit the container, it’s gone. Of course, just because the container is missing doesn’t mean it doesn’t exist,DockerThe container has been saved automatically. If the setup container name is not displayed at creation time, you can find it in the following waysLost container

#List recently created containers
$: docker container ls -n 1
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES
c028c091f964        busybox:latest      "sh"                13 minutes ago      Exited (0) 27 seconds ago                       upbeat_cohen

#Query container details
$: docker container inspect c028c091f964
...

1.1.2 submit changes to generate images

It’s easy to build the image manually. First find the changed container object and submit the change. After the commit is completed, the image is generated. However, the image at this time only has an automatically generated serial number that uniquely identifies it. In order to facilitate the retrieval of images, it is necessary to name and label the images.

The command line operations are as follows:

#Submit the change and build the image
$: docker commit -a JayL -m "add newfile" c028c091f964
sha256:01603f50694eb62e965e85cae2e2327240e4a68861bd0e98a4fb4ee27b403e6d

#To name the image, the original image ID can take the first few bits
$: docker image tag 01603f50694eb62e9 busybox:manual

#Verify new image
$: docker run busybox:manual ls -al newfile
-rw-r--r--    1 root     root             0 Jun 15 05:25 newfile

Through the above two steps, the process is completedDockerThe image is created manually. It’s very simple, isn’t it. But it is also very troublesome. You must first create a new container, commit changes, and generate images. The whole process can be scripted, which is also what the next section will say, automatic buildDockerMirror image.

1.2 automatic construction

1.2.1 construction of dockerfile

Automated buildDockerMirror image,DockerInstead of shell script, the company provides a set of independent syntax to describe the whole construction process. The file edited by this syntax is calledDockerfile。 Building images automatically is by writingDockerfileFile built.

Also complete the above work withDockerfileIt says:

FROM busybox:latest
RUN  touch /newfile

As for more detailedDockerfilePlease refer to the official guide for grammar.

completeDockerfileAfter writing, the build is triggered by a command. The whole process is scripted as follows:

$: mkdir autobuild && cd autobuild
$: cat <<EOF > Dockerfile
FROM busybox:latest
RUN  touch /newfile
EOF
$: docker build -t busybox:autobuild .

2 image storage

2.1 composition of image

The docker image consists of a set ofread-onlyMirror layer ofImage LayerComposed of. The docker container is based on the docker image, adding a layer: container layerContainer Layer。 Container layerContainer Layeryesread-write Yes. If theContainer LayerconductcommitCommit operation, this layer becomes a new mirror layerImage Layer。 newDocker ImageIt’s built.

$: mkdir layer && cd layer && touch newfile
$: cat <<EOF > Dockerfile
FROM scratch
ADD  newfile .
EOF
$: docker build -t layer .

The following figure provided on the official website can clearly see the connection and difference between image and container:

Build a safe, reliable and minimized docker image

The composition of a specific imageLayerYou can query by the following command:

#Construction layer history of mirror image
$: docker history busybox:autobuild
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
845cc5130d2c        17 minutes ago      /bin/sh -c touch /newfile                       0B
ef46e0caa533        4 days ago          /bin/sh -c #(nop) CMD ["sh"]                   0B
<missing>           4 days ago          /bin/sh -c #(nop) ADD file:1067e5a... in /  1.21MB

It’s not hard to see, mirror imagebusybox:autobuildA total of three bottom-up layer builds were performed. The specific construction instructions can be obtained through the command in the third column.<missing>This layer is built on other systems and is not available locally. Just ignore it.

2.2 Union FileSystem

To understand the storage of docker image, you must first understandJoint file systemUnionFS (Union File System)The so-calledUnionFSIt’s merging directories from different physical locationsmountTo the same directory.UnionFSThere are many specific implementations of:

  • Early UFS
  • AUFS
  • OverlayFS

    • overlay
    • overlay2

specificDockerWhich is used on the hostUnionFSThe file system driver can be queried by the following command:

$:  docker info | grep Storage
Storage Driver: overlay2

overlay2Is a more modern federated file systemUnionFS, which is better thanoverlayThe early versions of have been greatly improved in stability and performance. So the latestDockerAll the storage drivers used areoverlay2

For the convenience of demonstrationUnionFSFile system, if it is a MacOS system, it is recommended to installDocker MachineOpen a new virtual machine operation, excludingDocker for MacOSAll kinds of environment interference running on virtual machine. specificDocker MachinePlease refer to the relevant documents for the installation of.

First create a newDocker Machine:

#Create
$: docker-machine create ufs
...
Docker is up and running!

#Log in
$: docker-machine ssh ufs
... ok

#Query overlay
$: cat /proc/filesystems | grep overlay
nodev    overlay

By confirmation, this oneDocker MachineIt’s supportUnionFSFile system, usingoverlayStorage driver. sinceUnionFSnamelyMerge directories from different physical locations into the same directory. now let’s implement it on the command lineDockerProvided on the official websiteUnionFSSchematic diagram of.

Build a safe, reliable and minimized docker image

As can be seen from the figure, we need to provide two directories, which representContainer LayerandImage Layer。 Directory name, take the name on the right side of the diagram:

  • catalogupperOn behalf ofContainer Layer
  • cataloglowerOn behalf ofImage Layer

In addition to these two directories, theUnionFSTwo more directories are needed to mount the directory:

  • catalogmerged, represents the mounted directory, that is, the merged directory
  • catalogwork, must be an empty directory, yesoverlayThe working directory required for the storage driver mount.

The folder structure in the diagram is realized through the command line

#Create a test directory
$: mkdir demo && cd demo

#Create subdirectories and files
$: mkdir upper lower merged work
$: touch lower/file1 lower/file2 lower/file3
$: touch upper/file2 upper/file4

#Distinguish the following File2 by file content
$: echo lower > lower/file2
$: echo upper > upper/file2

#Not mounted
$: ls merged

So far, everything has been normal file system operations. It is now approvedmountCommand to proceedUnionFSDirectory mount of file system

#Directory merge mount to merged
$: sudo mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work merged

#After mounting
$: ls merged
file1 file2 file3 file4

#File2 uses the File2 file of the top-level upper
$: cat merged/file2
upper

Next, we will use theAddition, deletion and modificationDeepen rightUnionFSUnderstanding of file system:

  • New documents
#New documents
$: touch merges/file5
$: ls merged/
file1  file2  file3  file4  file5
#New documents写在顶层的 upper 文件夹
$: ls upper/
file2  file4  file5
$: ls lower/
file1  file2  file3
  • Modifying documents
#Modification of document cow Technology
$: echo mod > merged/file1
$: ls upper/
file1 file2  file4  file5
$: cat upper/file1
mod
$: cat lower/file1
  • Delete file
#Delete file
$: rm merged/file1
$: ls -al upper | grep file1
c---------    1 root     root        0,   0 Jun 17 10:41 file1
$: ls -al lower | grep file1
-rw-r--r--    1 docker   staff            0 Jun 17 10:15 file1

The actual operation completes the above process, I believe that you areUnionFSFile system has a more intuitive feeling. You might ask,Docker ImageThe underlying image of is composed of a set ofLayerComposed of multiple underlying directories inUnionFSHow to mount? In fact, it is very simple, just need to pass:Separate them.

#Multi level directory: lower1 / lower2 / lower3
$: sudo mount -t overlay overlay -olowerdir=lower1:lower2:lower3,upperdir=upper,workdir=work merged

After the mount is completed,lower1 / lower2 / lower3Readers can test the order of the stack.

Finally, let’s check the system mount list,

mount | grep overlay
overlay on /home/docker/demo/merged type overlay (rw,relatime,lowerdir=lower,upperdir=upper,workdir=work)

From the existing output, we can see that at present wedocker-machineOnly one is attached inoverlaycatalog.

2.3 storage of image

Now we’re on this newdocker-machineBuild a1.2Described inDockerImage:busybox:autobuild

$: mkdir autobuild && cd autobuild
$: cat <<EOF > Dockerfile
FROM busybox:latest
RUN  touch /newfile
EOF
$: docker build -t busybox:autobuild .

#After the construction, there are two docker images in the system
$: docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             autobuild           2e32da74b3ad        4 seconds ago       1.22MB
busybox             latest              e4db68de4ff2        2 days ago          1.22MB

After the build is complete, let’s take a look directly at itdocker-machineMount of file system on:

#Docker runs without container
$: mount
...
/dev/sda1 on /mnt/sda1/var/lib/docker type ext4 (rw,relatime,data=ordered)

#When docker runs container
#Restart the new session and run a container instance ` docker run - it ' busybox:autobuild sh `
$: mount
...
/dev/sda1 on /mnt/sda1/var/lib/docker type ext4 (rw,relatime,data=ordered)
overlay on /mnt/sda1/var/lib/docker/overlay2/a54541dd24971b9491a54b43cdf51f4ef9c87c1cd29748bb3fe64dedafd91b56/merged type overlay (rw,relatime,lowerdir=/mnt/sda1/var/lib/docker/overlay2/l/KLGL6INSJ2UBLMAUP5B4IORUTG:/mnt/sda1/var/lib/docker/overlay2/l/BGIT3WQZVII4Z2THF35I6T5V5O:/mnt/sda1/var/lib/docker/overlay2/l/6GZ2NT4UQT6EQK3IT4IGMBXU4T,upperdir=/mnt/sda1/var/lib/docker/overlay2/a54541dd24971b9491a54b43cdf51f4ef9c87c1cd29748bb3fe64dedafd91b56/diff,workdir=/mnt/sda1/var/lib/docker/overlay2/a54541dd24971b9491a54b43cdf51f4ef9c87c1cd29748bb3fe64dedafd91b56/work)
shm on /mnt/sda1/var/lib/docker/containers/e50f19c5bde3fe53cde3729de92f75b74323f7ebb506b0635eb76dd5b81e080a/mounts/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)
nsfs on /var/run/docker/netns/3c464f8003e8 type nsfs (rw)

Compared with the output, it can be clearly seen that only focus on the momentoverlayMounting condition. obtain:

  • The mounted directory is:
    /mnt/sda1/var/lib/docker/overlay2/a54541dd24971b9491a54b43cdf51f4ef9c87c1cd29748bb3fe64dedafd91b56/merged
  • The container layer is:
    /mnt/sda1/var/lib/docker/overlay2/a54541dd24971b9491a54b43cdf51f4ef9c87c1cd29748bb3fe64dedafd91b56/diff
  • The mirror layer is:
    /mnt/sda1/var/lib/docker/overlay2/l/KLGL6INSJ2UBLMAUP5B4IORUTG
    /mnt/sda1/var/lib/docker/overlay2/l/BGIT3WQZVII4Z2THF35I6T5V5O
    /mnt/sda1/var/lib/docker/overlay2/l/6GZ2NT4UQT6EQK3IT4IGMBXU4T

The mirror layer uses soft connection. We can communicate the same informationdocker inspectFind out.

$: docker inspect <container-id> -f '{{.GraphDriver.Data.MergedDir}}'
$: docker inspect <container-id> -f '{{.GraphDriver.Data.UpperDir}}'
$: docker inspect <container-id> -f '{{.GraphDriver.Data.LowerDir}}'

The output path is concreteDockerThe storage location of the mirror.

3. Minimize docker image

3.1 why minimize docker image

minimizeDockerThe reasons for mirror image can be summarized as follows:

  • save money, reduce network traffic and save mirror storage space
  • Save time, accelerate image deployment time
  • security, limited function reduces the possibility of being attacked
  • environment protectionWaste is classified. It’s shameful to waste resources

3.2 how to build a minimized docker image

Press1.31.4The principle and storage of the image discussed in minimizeDockerThere are two main ways to mirror image:

  • Reduce the layer size of the mirror
  • Reduce the number of layers of the mirror

Start with the simpleReduce the number of layers in the mirror layerStart.

3.3 reduce the number of layers in the mirror image

3.3.1 combined command

In definitionDockerfileEach instruction corresponds to a new mirror layer. adoptdocker historyThe command can find out the detailsDockerThe layers built by the image and the instructions used by each layer. In order to reduce the number of layers of the image, when the image is actually constructed, the&&The execution process of the connection command defines multiple commands to be executed in one build instruction. For example:

FROM debian:stable

WORKDIR /var/www

RUN apt-get update && \
    apt-get -y --no-install-recommends install curl \
        ca-certificates && \
    apt-get purge -y curl \
        ca-certificates && \
    apt-get autoremove -y && \
    apt-get clean

3.3.2 compressed image layer

In addition to passing multiple commands through&&Connect to a build instruction outside theDockerDuring the construction of the image, you can also use the--squashThe image layer can be compressed into a new image layer.

The specific orders are as follows:

$: docker build --squash -t <image> .

3.4 reduce the mirror layer size

3.4.1 select basic image

To reduce the size of the layer, you need to start from the beginning, that is, what kind of base image is selected as the initial image. In general, you will start with the following three basic images.

  • Mirror scratch(empty image), size 0b
  • Mirror busybox(empty image + busybox), size 1.4MB
  • Mirror Alpine(empty image + busybox + APK), size 3.98mb

Mirror busyboxadoptbusyboxThe program provides some basic Linux operating commands,Mirror AlpineOn the basis ofapkPackage management command, easy to install all kinds of tools and dependent packages. The widely used images are basicallyMirror AlpineMirror busyboxIt is more suitable for some fast experimental scenes. andMirror scratchEmpty image, because it does not provide any auxiliary tools, is suitable for programs that do not rely on any third-party library. becauseMirror scratchThe empty image itself does not provide anycontainer OSSo the program runs inDocker HostOn the host computer, it’s just usingDockerTechnology provides isolation technology.

Careful readers may find that in theMacOSThe program compiled on theMirror scratchWhen the image is empty, the container will report an error. That’s because,Docker for MacIs running inLinuxOn the virtual machine. So it can’t be built directlyMacOSThe executable program inDocker for MacThe system runs as an empty image.

3.4.2 multi stage image building

Multi stage constructionMulti-Stage BuildyesDocker 17.05New features introduced at the beginning of the release. By dividing the image query which was built in only one phase into several phases. The reason why multi-phase image building can reduce the size of the image is that the dependent packages and temporary files related to the publisher at compile time are not needed for the final release of the image. By dividing different stages and building different images, the final image depends on what entities we really need to publish.

FROM golang:1.11-alpine3.7 AS builder

WORKDIR /app
COPY main.go .
RUN go build -o server .

FROM alpine:3.7

WORKDIR /app
COPY --from=builder /app .

CMD ["./server"]

AboveDockerfileIt’s multi-stage constructionbuilderThe basic image used by the phase isgolang:1.11-alpine3.7, obviously because of the need of compile time, for the release of realserverThe procedure is completely unnecessary. Through the multi-stage construction of the image, we can only package the required entities to form the image.

In addition to the multi-phase build, if you want to ignore some redundant files in the image, you can also use the.dockerignoreThe method is defined in the file. Functions and.gitignoresimilar.

4. Reinforce docker image

minimizeDockerThe construction of the image is finished, but our work is not finished. We also need to doreinforcehandle.

4.1 mirrored content addressable identifier (caiid)

The content addressable image identifiers can be used to verify the source basic image content to ensure that it has not been tampered with by a third party. The specific operation mode is to build their own image at the same time, the content of the basic image contentsha256The summary value is set to prevent tampering without knowledge.

First of all, the concrete image is correctsha256Summary value

#Query the sha256 summary of the specific image through the command
$: docker inspect busybox:autobuild -f "{{.RepoDigests}}"
sha256:9b63a0eaaed5e677bb1e1b29c1a97268e6c9e6fee98b48badf0f168ae72a51dc

AgainDockerfileWhen defining, set thesha256Summary value

FROM [email protected]:9b63a0eaaed5e677bb1e1b29c1a97268e6c9e6fee98b48badf0f168ae72a51dc
...

Note: the retrieval of the addressable identifier of the image content must go through a push or pull operation, that is, after it is published on the image registration service, the result can be found through the above inspection command. If it is only a local image, it cannot be obtained through the inspect command. Of course, it is only a local image, and the addressable identifier of image content is unnecessary.

4.2 user rights

Once a container is created, its default users can be set in the image. By setting the necessary default users for mirroring, you can restrict their execution rights in the container. To some extent, it also improves the security level of the mirror. However, this needs to be set according to the specific business publishing situation. Under normal circumstances, the basic image is still the root user as the default user

Safety principle:The image itself is customized for a specific application. By default, the user permissions should be reduced as much as possible.

4.3 suid and sgid

In addition to setting the necessary default user for the image itself, there will be a kind of program in the image, even if it is executed by ordinary users, but it will be executed with a higher level of authority at runtime. This is the special permission of suid and sgid provided by the system for executable files and directories.

By setting suid or sgid attributes on the executable file, the user who originally executed the command will switch to the owner of the command or the permissions of the group to execute the command. In other words, the authority to execute commands is enhanced.

In the actual image construction, we should try to avoid the possible vulnerabilities caused by such privilege promotion. It is recommended to scan the image for such executable files during image construction, and delete them as much as possible if any. For the delete command, please refer to:

#Scan and delete executable files with special permission during image construction
RUN for i in $(find / -type f \( -perm +6000 -o -perm +2000 \)); \
    do chmod ug-s $i; done

5. Review docker image

just asCode ReviewSimilarly, code review can greatly improve the quality of enterprise projects. Container image is also the product of developers or operation and maintenance personnel, so it is necessary to review it.

Although we can passdockerCommand combined with file system browsing to review container image, but the process requires manual participation, which is difficult to achieve automation, let alone integrate the image review into the CI process. But a good tool can help us do that.

Recommend a very good open source project dive, please refer to its project page for specific installation. It is not only convenient for us to query the detailed information of the specific image layer, but also can be used as theCIUse of mirror review during continuous integration. Using it can greatly improve the speed of our review of images, and can make this process automated.

The specific dynamic operation diagram of the project is as follows:

Build a safe, reliable and minimized docker image

If you want to review the image, you can perform the following command operations:

$: CI=true dive <image-id>
Fetching image... (this can take a while with large images)
Parsing image...
Analyzing image...
  efficiency: 95.0863 %
  wastedBytes: 671109 bytes (671 kB)
  userWastedPercent: 8.2274 %
Run CI Validations...
  Using default CI config
  PASS: highestUserWastedPercent
  SKIP: highestWastedBytes: rule disabled
  PASS: lowestEfficiency

From the output information can get a lot of useful information, integratedCIThe process is very easy.diveIt provides.dive-ciAs projectCIto configure:

rules:
  # If the efficiency is measured below X%, mark as failed.
  # Expressed as a percentage between 0-1.
  lowestEfficiency: 0.95

  # If the amount of wasted space is at least X or larger than X, mark as failed.
  # Expressed in B, KB, MB, and GB.
  highestWastedBytes: 20MB

  # If the amount of wasted space makes up for X% or more of the image, mark as failed.
  # Note: the base image layer is NOT included in the total image size.
  # Expressed as a percentage between 0-1; fails if the threshold is met or crossed.
  highestUserWastedPercent: 0.20

Integration intoCIAdd the following command:

$: CI=true dive <image-id> 

Mirror review, like code review, is something you can’t stop after you start resisting it. It’s a matter of time. For enterprises and individuals, both profit and no harm.

6. Reference resources

  • the-overlay-filesystem
  • how-to-build-a-smaller-docker-image
  • dive
  • overlayfs
  • Manage data in Docker
  • overlayfs-driver
  • container and layers
  • The 3 Biggest Wins When Using Alpine as a Base Docker Image
  • Understanding Docker “Container Host” vs. “Container OS” for Linux and Windows Containers