Breaking through the limitation of dockerhub, full image acceleration service

Time:2020-12-1

Recently, dockerhub has revised the pricing, which limits 200 pull / 6 hours for free accounts and 100 pull / 6 hours for anonymous accounts. In this article, I will introduce how to use cache to solve this problem.

background

Dockerhub is the earliest and largest container image warehouse in the world, hosting many docker images of operating system distributions and various software.

In the process of promoting business containerization, it is inevitable that we will need to use container images from dockerhub.Whether it is used in personal local environment or for running test service

Here are two main solutions:

  • Build some public basic images and store them in the enterprise’s private image warehouse for the business side to use

    In this scenario, if the business side needs some niche / non basic images occasionally, which may only be used for temporary testing, it is not necessary to maintain such images as basic images under normal circumstances.

    The results may be:

    • When the network is in poor condition, it is endless waiting;
    • Pull the image first, and thendocker tagAfter retag, push it to the private image warehouse of the enterprise. In this case, if there is no good image management rules, there will be all kinds of meaningless images in the image warehouse, resulting in a waste of storage resources.
  • Configure proxy for docker daemon to accelerate:

    • For many domestic image acceleration services, only the official docker image acceleration service is provided, while the personal / organizational image does not
    • Even if the same image is downloaded from different nodes, it still needs to be accelerated through the network, resulting in additional overseas bandwidth cost;

Recently, dockerhub has been modifiedIts service priceFor free users, the following restrictions are imposed:

  • Users who are not logged in can only pull 100 times every 6 hours
  • Logged in users can only pull 200 times every 6 hours

If we continue to use the above two modes, because the export IP address is relatively fixed, it is easy to trigger the quota limit of dockerhub. The restrictions will be fully implemented on November 1.

In order to be able toImprove efficiency, andSave the cost of accelerating bandwidthEnterprises / individuals need a dockerhub full image acceleration service, as we often saypull through cache

Let me introduce how to use the docker open source projectregistry:2To meet this demand.

Start service

useregistry:2It is very simple to deploy the image cache service, which is executed firstdocker pull registry:2Download the required image:

(MoeLove) ➜  docker pull registry:2
2: Pulling from library/registry
cbdbe7a5bc2a: Pull complete 
47112e65547d: Pull complete 
46bcb632e506: Pull complete 
c1cc712bcecd: Pull complete 
3db6272dcbfa: Pull complete 
Digest: sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d
Status: Downloaded newer image for registry:2
docker.io/library/registry:2

To minimize the configuration of the dockerhub image cache service, only one configuration item is requiredREGISTRY_PROXY_REMOTEURLThen:

Here, by the way, I created a separate one calledhub-cacheAnd the corresponding volume is created.

(MoeLove) ➜  ~ docker network create hub-cache
19a39f873a23150d3bdaf021e040ccccb092ee3071884d64d52a92df0397b220
(MoeLove) ➜  ~ docker volume create hub-cache
hub-cache
(MoeLove) ➜  ~ docker run --name=cache -d --restart=always --network=hub-cache -v hub-cache:/var/lib/registry -p 5000:5000  -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io registry:2 
6cbdcbdcc2d62ec781479901c20be43184a48b2d73e06f04bd4693253c0c5a73
(MoeLove) ➜  ~ docker ps -l
CONTAINER ID   IMAGE        COMMAND                  CREATED         STATUS         PORTS                    NAMES
6cbdcbdcc2d6   registry:2   "/entrypoint.sh /etc…"   8 seconds ago   Up 6 seconds   0.0.0.0:5000->5000/tcp   cache

Verify acceleration effect

Start a new docker in docker container for validation, so as to avoid the impact of local environment.

By passing on--registry-mirror http://cache:5000Set the registry you just started to mirror.

(MoeLove) ➜  ~ docker run  --network=hub-cache -d --privileged docker:dind --registry-mirror http://cache:5000 
73c56ac25d68927c9f5b0e458f2babc0699cf8595df0d1e86c021fd03d477384
(MoeLove) ➜  ~ docker exec -it $(docker ps -ql) sh
/ා check whether the configuration is in effect
/ # docker info --format '{{ .RegistryConfig.Mirrors }}' 
[http://cache:5000/]

/ # time docker pull prom/prometheus
Using default tag: latest
latest: Pulling from prom/prometheus
76df9210b28c: Pull complete 
559be8e06c14: Pull complete 
6a4bb3319487: Pull complete 
2cca90a64593: Pull complete 
d2014e464a99: Pull complete 
70b42590e4a2: Pull complete 
54645fcbd6cc: Pull complete 
67d9943de656: Pull complete 
b9c749b1af90: Pull complete 
9723d8eb5323: Pull complete 
7d20502d5322: Pull complete 
3e519cce6f63: Pull complete 
Digest: sha256:d43417c260e516508eed1f1d59c10c49d96bbea93eafb4955b0df3aea5908971
Status: Downloaded newer image for prom/prometheus:latest
docker.io/prom/prometheus:latest
real    0m 42.71s
user    0m 0.12s
sys     0m 0.09s


/ # docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
prom/prometheus     latest              cdfc440228d0        8 days ago          168MB

/ # docker rmi prom/prometheus                                     
Untagged: prom/prometheus:latest
Untagged: prom/[email protected]:d43417c260e516508eed1f1d59c10c49d96bbea93eafb4955b0df3aea5908971
Deleted: sha256:cdfc440228d01d7a94937d7a047fa6461efc1b1806bb20677043fee032810830
Deleted: sha256:c72f348fd2f923996ea80222feb77e34aba9de397bd96206ddc3c8651adc306d
Deleted: sha256:e34df2c00334266a67bb846b958ba6eae3b1d5cdfe9d763707027a23e7c85100
Deleted: sha256:d2cb38310ada122064b7333bbfc12c67dc58acb30e29146b3ba1e24adc27a950
Deleted: sha256:7a87cd520d19a83b3582541aac4d95098ae5016b092e72eaf80dc54f587bf51e
Deleted: sha256:f84c79dceed6b5a27234c1291d0bdccab5c459d587f13934d74db9b9e79471c6
Deleted: sha256:f542b0cffe0fe16c31c98e7eed934d5fea5e598c03b53b4efd308a62e0e9c6a9
Deleted: sha256:f746b4a525727bcb79367d009d707ef45d75bac09aaa18a68c20a19046df0897
Deleted: sha256:09b45653ee7062c7cd754885bf46ebe554d0794573fb2e200acea8644e64670f
Deleted: sha256:867526c56b30e67493341ef33890aa242c1131e4bb4151e60011b4d450892d59
Deleted: sha256:86d629b358ee70bdb0f0a11c10915b8551e904fe337f9a8bfcad476977329532
Deleted: sha256:842455c528af7383ba4a0de424fc63664a0248581a191516d6dbf45195c69426
Deleted: sha256:1be74353c3d0fd55fb5638a52953e6f1bc441e5b1710921db9ec2aa202725569

/ # time docker pull prom/prometheus
Using default tag: latest
latest: Pulling from prom/prometheus
76df9210b28c: Pull complete 
559be8e06c14: Pull complete 
6a4bb3319487: Pull complete 
2cca90a64593: Pull complete 
d2014e464a99: Pull complete 
70b42590e4a2: Pull complete 
54645fcbd6cc: Pull complete 
67d9943de656: Pull complete 
b9c749b1af90: Pull complete 
9723d8eb5323: Pull complete 
7d20502d5322: Pull complete 
3e519cce6f63: Pull complete 
Digest: sha256:d43417c260e516508eed1f1d59c10c49d96bbea93eafb4955b0df3aea5908971
Status: Downloaded newer image for prom/prometheus:latest
docker.io/prom/prometheus:latest
real    0m 5.27s
user    0m 0.06s
sys     0m 0.03s

As you can see, in the first pullprom/prometheusIt takes 42 + s time to image, but it only takes 5 + s to pull again after deleting the downloaded image. The speed increase is very obvious. Image acceleration effect achieved

Use configuration

For Linux systems, you only need to/etc/docker/daemon.jsonWrite the domain name of your image acceleration service in the file (if there is no such file, create it directly), and restart docker daemon (you can also choose reload configuration).

{
        "registry-mirrors": [
                "https://hub-cache.moelove.info"
        ]
}

Or add it to the startup parameter of docker daemonregistry-mirrorConfiguration item.

For Mac and windows users, configure registry mirrors directly in docker desktop system settings.

be carefulIf theHTTP_PROXYorHTTPS_PROXYThen you need to configure the accelerated domain name in theNO_PROXYAvoid being represented.

summary

This article describes how to use docker open sourceregistry:2Build the image acceleration service of dockerhub. Only the simplest configuration is described here.

But if deployed in an enterprise environment, more configuration is needed. For example, you can configureREGISTRY_HTTP_DEBUG_PROMETHEUS_ENABLEDPrometheus metrics is exposed to monitor service availability and view the effect of cache; logs and related fields can be configured;
In order to avoid triggering traffic restrictions to dockerhub after November, you can expand the capacity horizontally, prepare multiple export IP addresses, and configure accounts.

Recently, the newly released harbor v2.1 seems to have one moreAsproxy cacheCharacteristics ofBut it is not the same as thepull through cacheDifferent. Using the proxy cache feature of harbor, you need to set the image to be pulled to<harbor_servername>/<proxy_project_name>/repo/name:tagIt is not convenient, but it is also a very good feature.


Welcome to subscribe my official account number [MoeLove].

Breaking through the limitation of dockerhub, full image acceleration service

Recommended Today

020_CSS3

catalog How to learn CSS What is CSS History of development quick get start Advantages of CSS Three ways to import CSS Expansion: two ways of writing external style selector Basic selector Hierarchy selector Structure pseudo class selector attribute selectors Beautify web page elements Why beautify web pages Span label: for the text that needs […]