Docker cannot stop or delete the solution to the container service problem

Time:2022-5-6

preface

Today, a development classmate gave me feedback that a container service cannot stop, RM (docker RM – F) and kill operations, that is, the container service cannot be terminated ~

Operation steps

(1) The directory of docker cannot be deleted by executing the delete command:


# ll /var/lib/docker/containers | grep caf8ef20f3c1
# cd /var/lib/docker/containers 
# rm -rf caf8ef20f3c1c78f03a5844ee23abc1d7e44246f242292040f1ef288899d0cb8

At this time, we will receive such an error report:

RM: unable to delete “/ var / lib / docker / containers / caf8ef20f3c1c78f03a5844ee23abc1d7e44246f24292040f1ef288899d0cb8 / secrets”: device or resource busy

Cannot delete “/ var / lib / docker / containers / caf8ef20f3c1c78f03a5844ee23abc1d7e44246f24292040f1ef288899d0cb8 / SHM”: device or resource busy

(2) From the above error report, we can see that “secrets” and “SHM” cannot be deleted due to shared mounting. First find the mounting location, then cancel the mounting, and then delete:


# cat /proc/mounts |grep "docker" |grep "caf8ef20f3c1"

(3) Unmount:


# umount /data/sys/var/docker/containers/caf8ef20f3c1c78f03a5844ee23abc1d7e44246f242292040f1ef288899d0cb8/secrets
# umount /data/sys/var/docker/containers/caf8ef20f3c1c78f03a5844ee23abc1d7e44246f242292040f1ef288899d0cb8/shm

(4) View again:

#Cat / proc / mounts | grep "docker" | grep "caf8ef20f3c1" // no more

(5) Now delete the docker Directory:


# cd /var/lib/docker/containers 
# rm -rf caf8ef20f3c1c78f03a5844ee23abc1d7e44246f242292040f1ef288899d0cb8

(6) Delete container service

Now let’s use RM or kill to delete the container service:


# docker rm -f caf8ef20f3c1c

or


# docker kill --signal=SIGINT caf8ef20f3c1

If hang occurs after the above command is run, restart the docker service:


# systemctl restart docker

After completing the above steps, the problems encountered can be basically solved ~

Add: the docker container cannot be stopped or killed

Problem process

A MySQL container in an environment cannot be stopped or killed or RM

Sudo docker PS | grep MySQL view the container


7844250860f8 mysql:5.7.22  "/.r/r docker-entr..."  41 minutes ago  Up 8 minutes  r-dlrel-mysql-1-66df8f33

After using the docker stop / docker kill / docker RM – F and other commands, the container will restart automatically immediately

Check the container immediately. The running time is: up less than a second, indicating that the container is started immediately


7844250860f8 mysql:5.7.22  "/.r/r docker-entr..."  42 minutes ago Up Less than a second  r-dlrel-mysql-1-66df8f33

Kill the physical process corresponding to the container and still restart automatically

Get physical process mode: 1 State in docker inspect The PID field is the physical process ID; 2. PS command

Check the container restart policy. If the policy is no, it will not restart automatically

If you need to update the restart policy of the running container, you can use this command: docker update – restart = no my container


"RestartPolicy": {
  "Name": "no",
  "MaximumRetryCount": 0
},

Magical problem solving between programmers

Have you ever seen such a scene:

When I came to my colleagues and explained the problem clearly, I suddenly realized it.

The problem is very simple, but there is a problem when the program runs. Then find a colleague to help check the basic configuration, and you have an epiphany.

This time I belong to the first type. Just after finishing the problem, I immediately remembered: wipe, it is the container scheduling tool rancher that is scheduling. After the container hangs, it will restart automatically.

Log in to rancher and see if so, “Oolong” problem. Although it’s not a problem this time, docker does have a problem that it can’t stop, and there are a lot of data.

The above is my personal experience. I hope I can give you a reference, and I hope you can support developpaer. If there are mistakes or not fully considered, please don’t hesitate to comment.