Kubernetes, or k8s for short, is an open source project released by Google in 2014.
What problems did kubernetes solve?
A real production environment application will contain multiple containers, which are likely to be deployed across multiple server hosts. Kubernetes provides the ability to choreograph and manage containers for large-scale deployment of those workloads. Kubernetes choreography allows you to build multi container application services, schedule or scale these containers on a cluster, and manage their health over time.
- Kubernetes Foundation
- Kubernetes optimization
- Kubernetes in action
There are many basic concepts of kubernetes. It’s easy to get dizzy when you first touch it. If you’re a novice, it’s recommended to look directly at the actual combat part and start first.
There are several important concepts in k8s.
|cluster||A k8s cluster|
|master||A machine in the cluster is the core of the cluster, responsible for the management and control of the whole cluster|
|node||A machine in the cluster is a workload node in the cluster|
|pod||K8s minimum scheduling unit, each pod contains one or more containers|
|label||A key = value key value pair can be attached to various resource objects to facilitate matching of other resources|
|controller||K8s manages pod through controller|
|service||Provide unified services to the outside world, and automatically distribute requests to the correct pod|
|namespace||Logically divide the cluster into multiple virtual clusters, each cluster is a namespace|
Cluster represents a k8s cluster, which is a collection of computing, storage and network resources. K8s uses these resources to run various container based applications.
Master is the brain of cluster, running services including Kube apiserver, Kube scheduler, Kube controller manager, etcd and pod network.
- Apiserver is the front-end interface of k8s cluster, which provides restful API. Various client tools and other k8s components can manage various resources in the cluster.
- Be responsible for deciding which node to run the pod on. When scheduling, the scheduler will fully consider the load of each node in the cluster, as well as the application requirements for high availability, performance, and data affinity.
- Responsible for managing various resources of the cluster to ensure the expected state of resource processing. Controller manager is composed of many controllers, including replication controller, endpoint controller, namespace controller, service account controller, etc.
- Different controllers manage different resources, such as: replication controller manages deployment, statefullset, damonset’s life cycle, and namespace controller resource manages namespace resources.
etcd(distributed key value repository)
- Responsible for saving the configuration information of the cluster and the status information of various resources. When the data changes, etcd will quickly notify the k8s related components.
- To be able to communicate with pod, cluster must deploy pod network. Flannel is one of the options.
The node node is where the pod runs. K8s components running on node nodes include kubelet, Kube proxy and pod network.
- Kubelet is the agent of a node. When the Kube scheduler in the master node determines to run the pod on a node, it will send the specific configuration information of the pod to the kubelet of the node. Kubelet creates and runs the container based on the information and reports the running status to the master node.
- Each node runs the Kube proxy service, which is responsible for forwarding requests to the backend container. If there are multiple copies, the Kube proxy implements load balancing.
- To be able to communicate with each other, k8s cluster must deploy pod network. Flannel is one of the options.
Each pod contains one or more containers, and the containers in the pod are dispatched to the node node by the master as a whole.
Why does k8s use pod to manage containers rather than directly manipulate them?
1. Because some containers naturally need to be closely connected, and put in a pod to represent a complete service, k8s will add pause containers to each pod at the same time to manage the state of internal container groups.
2. All containers in pod share IP and volume, which facilitates communication and data sharing between containers.
When do I need to define multiple containers in a pod?
A: these containers are very closely related and need to share resources directly, such as a crawler and a web server program. Web server relies heavily on crawlers to provide data support.
Label is a key value pair with key = value, where both the key and value are specified by the user.
Usage scenarios of label:
- Through the label selector, the pod copies to be monitored are filtered out, so that the number of pod copies meets the expectation.
- Select the corresponding pod through the label selector, and automatically establish the request forwarding routing table from each service to the corresponding pod, so as to realize the intelligent load balancing mechanism of the service.
- By defining label for node and using node selector matching in pod, Kube scheduler process can realize the characteristics of pod oriented scheduling.
In a word, label can create multiple groups of labels for objects. Label and label selector jointly build the core application model of k8s system, which enables the managed objects to be managed in groups and realizes the high availability of the whole cluster.
K8s usually does not create a pod directly, but manages the pod through the controller. The controller defines the deployment features of pod, such as how many copies there are, what kind of node to run on, etc. In order to meet different business scenarios, k8s provides a variety of types of controllers.
- Most commonly used, you can manage multiple copies of a pod and ensure that the pod runs as expected, with the underlying calling replicaset.
- It is usually enough to use deployment to manage multiple copies of pod.
- For scenarios where each node runs at most one copy of the pod.
- Run storage daemons, such as glusterd or CEPH, on each node of the cluster.
- Run log collection daemons, such as fluent or logstash, on each node.
- Run monitoring on each node, such as the Prometheus node exporter or collected.
- It can ensure that the name of each copy of the pod is unchanged in the whole life cycle, while other controllers do not provide this function.
- It is used for applications that are deleted at the end of the run, while pod in other controllers usually runs continuously for a long time.
For the use case created by the deployment controller, if a pod hangs up, a new pod will be created automatically to maintain the number of pods in the configuration.
Containers can be divided into two categories according to the duration: Service containers and work containers.
Service containers usually provide services continuously and need to run continuously, such as HTTP server. Work containers are one-time tasks, such as batch programs, which exit after completion.
The deployment, replicaset and daemonset types in the controller are used to manage service class containers. For work class containers, we use job.
Service is a policy that can access a set of pods, usually called microservices. Which group of pods to visit is matched by label. Service provides load balancing for pod. The principle is to use iptables.
Why use service?
- Pods have a life cycle. They can be created or destroyed. However, once destroyed, the life will end forever. Pod may be created and destroyed frequently in a k8s cluster, and a new IP address will be generated for each reconstruction.
- Service logically represents a set of pods, which are selected by label. Service has its own IP, and this IP is the same. The client only needs to access the IP of service. K8s is responsible for establishing and maintaining the mapping relationship between service and pod. No matter how the pod changes, it will have no impact on the client, because service has not changed.
How to access the service on the Internet?
- Clusterip: expose the service through the internal IP of the cluster. Select this value. The service can only be accessed inside the cluster, which is also the default service type.
- Nodeport: exposes services through IP and port (nodeport) on each node. The nodeport service will be routed to the clusterip service, which will be created automatically. By request
nodeip:nodeport, you can access a service from outside the cluster.
- Loadbalancer: using cloud provider’s load balancer, services can be exposed externally. External load balancers can be routed to the nodeport service and the clusterip service.
- Externalname: by returning CNAME and its value, you can map the service to the contents of the externalname field (for example, foo.bar.example .com）。 No agent of any type has been created, which is only supported by Kube DNS of k8s 1.7 or later.
If multiple users use the same k8s cluster, how to separate the controller, pod and other resources they create?
A: use namespace.
If the physical cluster is logically divided into multiple virtual clusters, each cluster is a namespace, and the resources in different namespaces are completely isolated.
K8s creates two namespaces by default.
- Default if you do not specify when creating a resource, it will be placed in this namespace.
- Kube system stores the system resources created by k8s itself.
- health examination
- data management
- Password management
- Cluster monitoring
Powerful self-healing ability is an important feature of k8s container orchestration engine. The default implementation of self-healing is to restart the failed container automatically. In addition, users can also use liveness and readiness detection mechanisms to set up more detailed health checks, so as to achieve the following requirements:
- Zero downtime deployment
- Avoid deploying invalid mirrors
- More secure rolling upgrade
By default, k8s will consider that the container has failed and needs to be restarted only if the return value of the container process is non-zero. If we want to control the container restart more finely, we can use liveness and readiness.
The principle of liveness and readiness is to check regularly
/tmp/healthyIf the file exists, it means that the program has not failed. If the file does not exist, corresponding measures will be taken to give feedback.
Liveness takes the policy of restarting the container, while readness takes the policy of making the container unavailable.
If you need to restart the container in a specific situation, you can use liveness.
If you need to ensure that the container can always provide external services, you can use readiness.
We can use liveness and readness together, use liveness to determine whether the container needs to be restarted, and use readness to determine whether the container is ready to provide external services.
As mentioned above, pod may be destroyed and created frequently. When the container is destroyed, the data stored in the file system inside the container will be cleared. In order to persist the data of the container, k8s volume can be used.
The life cycle of a volume is independent of the container. The container in the pod may be destroyed and rebuilt, but the volume will be retained. In essence, a vloume is a directory. When a volume is mounted to a pod, all containers in the pod can access the volume.
Volume supports multiple types.
- The data is stored in the pod. For the container in the pod, it is persistent as long as the pod is still in the data.
- The data exists on the host, and the host is on the data.
AWS Elastic Block Store
- The data exists on the ECS.
- Customize an external storage space persistent volume, and then use persistent volume claim (PVC) to request space and store it when creating pod.
Volume provides various storage methods, but when using volume to read and write data, the container does not need to care whether the data is stored in the system of the local node or on the cloud disk. For containers, all types of volumes are just one directory.
The application may need sensitive information during startup, such as the user name and password to access the database. It is obviously inappropriate to store these information directly in the container image. The solution provided by k8s is secret.
Secret stores data as ciphertext, avoiding storing sensitive information directly in the configuration file. Secret will be mounted to pod in the form of volume. The container can use the sensitive data in secret in the form of file. In addition, the container can also use the data in the form of environment variables.
Create with profile mysecret.yaml :
apiVersion: v1 kind Secret metadata: name:mysecret data: username:admin password:123
After saving the profile, execute
kubectl apply -f mysecret.yamlTo create.
Use the created secret in Pod:
# mypod.yaml apiVersion: v1 kind: Pod metadata: name: mypod spec: containers: - name: mypod image: yhlben/notepad volumeMounts: - name: foo mountPath: 'etc/foo' readOnly: true volumes: - name: foo secret: secretName: mysecret
kubectl apply -f mypod.yamlCreate the pod and use secret. After creation, the secret is saved in the container under the directory / etc / foo / username, / etc / foo / password.
Creating k8s clusters and deploying containerized applications is just the first step. Once the cluster is running, we need to ensure that everything in the cluster is normal, which requires monitoring the cluster.
Common visual monitoring tools are as follows.
- Weave Scope
- Prometheus Operator
See the document for the specific steps, which will not be explained in detail here.
Through cluster monitoring, we can find problems in the cluster in time, but in order to facilitate further troubleshooting, we need to log.
Common log management tools are as follows.
- Elasticsearch is responsible for storing logs and providing query interfaces.
- Fluent is responsible for collecting logs from k8s and sending them to elastic search.
- Kibana provides a visual page where users can browse and search logs.
Kubernetes in action
Let’s actually deploy a k8s Notepad project, which is built with yhlben / Notepad image. After deployment, the image will provide a web version of Notepad service on port 8083 to view the demonstration.
In order to avoid the various pits in k8s installation, play with kubernetes is used for demonstration.
First, create three servers on play with kubernetes, with node1 as the master node and node2 and node3 as the working nodes. Next, carry out the following operations;
- Create a cluster
- Join node node
- Initialize cluster network
- Create controller
- Create service
- Execute deployment
Create a cluster
Running kubeadm init on node1 creates a cluster.
kubeadm init --apiserver-advertise-address $(hostname -i)
After execution, a token will be generated so that other nodes can join the cluster with this token.
Join node node
On the node2 and node3 machines, execute the following commands to join the node1 cluster.
kubeadm join 192.168.0.8:6443 --token nfs9d0.z7ibv3xokif1mnmv \ --discovery-token-ca-cert-hash sha256:6587f474ae1543b38954b0e560832ff5b7c67f79e1d464e7f59e33b0fefd6548
After the command is executed, node2 and node3 are added successfully.
View cluster status
On the node1 node, execute the following command.
kubectl get node
It can be seen that there are node1, node2 and node3 nodes in the cluster, but the three nodes are in notready state. Why?
A: because no cluster network has been created.
Create a cluster network
Execute the following code to create a clustered network.
kubectl apply -n kube-system -f \ "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 |tr -d '\n')"
After executing the command, wait a moment, and then check the node status. You can see that all three nodes in the cluster are ready.
We create the deployment through the configuration file and create a new one deployment.yaml The contents of the document are as follows:
#Version of profile format apiVersion: apps/v1 #Created resource type kind: Deployment #Metadata for resources metadata: name: notepad #Specification spec: #Define pod quantity replicas: 3 #Find the corresponding pod through label selector: matchLabels: app: mytest #Define template for pod template: #Metadata of pod metadata: #Define the label of pod labels: app: mytest #Describe pod specifications spec: containers: - name: notepad image: yhlben/notepad ports: - containerPort: 8083
After the file is created, execute the command:
kubectl apply -f ./deployment.yaml # deployment.apps/notepad created
View the deployed pod.
As you can see, we use the controller of deployment type to create three pods, which run on the node2 and node3 machines respectively. If there are more machines, it will also automatically load balance and distribute them to each machine reasonably.
Create service is similar to deployment, create new service.yaml The contents of the document are as follows:
apiVersion: v1 kind: Service metadata: name: my-service spec: #Deploy ports to access pod on nodes type: NodePort ports: - port: 80 #Port of service agent targetPort: 8083 #The access port provided to the outside of the cluster on the node node nodePort: 30036 #Match pod selector: app: mytest
After the file is created, execute the command:
kubectl apply -f ./service.yaml # service/my-service created
View creation results
Use kubectl get deployment and kubectl get service to view the creation results.
It can be seen that both deployment and service are created successfully, and it is known that the IP address exposed by the service is 10.106.74.65, and the port number is 80. Because the targetport is set, the service will automatically forward to the port 8083 corresponding to the pod when receiving the request.
Access deployment results
After the deployment is successful, we can access it in two ways:
- In the cluster: access through the clusterip + port port port of the service.
- Outside the cluster: access through any node + nodeport port.
#Intra cluster access curl 10.106.74.65 #Access outside the cluster #192.168.0.12 is the IP address of node2 node #192.168.0.11 is the IP address of node3 node curl 192.168.0.12:30036 curl 192.168.0.11:30036
Intra cluster access.
Access outside the cluster.
So far, the deployment has been successful. You must have doubts. It’s so troublesome to deploy such a simple web application. What’s the good about k8s? Let’s look down.
K8s operation and maintenance
The project has been deployed. Next, let’s practice an operation and maintenance, and feel the convenience brought by k8s.
Companies to do double 11 activities, need at least 100 containers to meet user requirements, how to do?
First of all, we should make use of the current server resources as much as possible to create more containers to participate in load balancing. Through docker stats, we can view the system resources occupied by containers. If the demand can’t be satisfied after full utilization, we can calculate how many machines we need to buy according to the remaining containers to realize the reasonable utilization of resources.
- Purchase the server, and join the server into the cluster as a node node.
- Execute the expansion command.
You can expand containers to 100 by executing the following command.
kubectl scale deployments/notepad --replicas=100
As shown in the figure, when 10 pods are expanded, node2 and node3 nodes share the load.
It can also be modified by deployment.yaml Replicas field in, executing
kubectl apply -f deployment.yamlTo execute the extension. If the activity is over, just delete the redundant servers and reduce the number of containers to restore the previous effect.
The double 11 campaign is very popular, but suddenly there is a demand, and it needs to be launched urgently. If rolling update is realized?
Rolling update is to update the code without downtime. Execute the following command to modify image.
kubectl set image deployments/notepad notepad=yhlben/notepad:new
It can also be modified by deployment.yaml Image field in, executing
kubectl apply -f deployment.yamlTo perform the upgrade.
If there is a problem with the update, k8s has a built-in command to restore the previous version:
kubectl rollout undo deployments/notepad
Through these two cases, k8s brings great convenience to operation and maintenance:Deploy projects quickly and efficiently，Support dynamic expansion、Rolling upgrade，At the same time, it can optimize the hardware resources on demand。
The purpose of this paper is to get started with k8s, which is achieved through a simple cluster, but there are also many pits, as follows:
Building projects with minikube
- It can quickly build a single node k8s environment.
- But in the local use of minikube to build a k8s cluster, many packages can not be packaged, nor can the global agent.
Using servers on Google cloud
- On gogole cloud, the network problem is solved, and all packages that need to be installed can be installed.
- However, because it is a new server, it needs various installation environments, such as docker, kubeadm, kubectl, etc. the installation process is tedious and may encounter errors.
- I don’t know which day I slipped my hands. The trial account became a paid account, and the bonus & dollar; 300 disappeared.
Use play with kubernetes
- The environment for k8s needs not to be matched. It has been installed, which is very good.
- The capacity of a single machine is too small. A slightly larger image cannot be installed.
- Occasionally, when play with kubernetes is used, an instance is kicked off the line just after it is created. In this case, it is impossible to play.
- No public IP is provided to verify the access to the Internet.
Finally, I would like to recommend the book “play kubernetes every five minutes”, which is very suitable for the introduction of k8s. Through a large number of simple practical operations, from easy to difficult, I really understand k8s, and a lot of theoretical knowledge in this paper also comes from this book.
For more articles, welcome to star and subscribe to my blog.