19. Kubernetes notes pod resource scheduling (II) podaffinity, podantiaffinity pod affinity and anti affinity

Time:2022-5-26

summary:

  • Let’s first understand the topology key field
    Pod affinity scheduling requires that all related pod objects run in the “same location”, while anti affinity scheduling requires that they cannot run in the “same location”,
    The “same location” specified here is defined by topologykey. The corresponding value of topologykey is a label name on the node. For example, each node has a label of zone = a, each node has a label of zone = B, and the pod affinity topologykey is defined as zone. Then, when scheduling pod, it will be scheduled around topology A and topology B, and the node under the same topology will be “same location”.
  • As the name suggests, topology means topology
    This refers to a topology domain, which refers to the concept of a range, such as a node, a cabinet, a computer room or a region (such as Hangzhou and Shanghai). In fact, it corresponds to the label on the node. The topologykey here corresponds to the key of the tag on the node (no value). It can be seen that the topologykey is actually used to filter nodes. In this way, we can dispatch each pod across clusters, machine rooms and regions.

19. Kubernetes notes pod resource scheduling (II) podaffinity, podantiaffinity pod affinity and anti affinity

View the detailed description of podaffinity

[[email protected] Scheduler]# kubectl explain pods.spec.affinity
KIND:     Pod
VERSION:  v1

RESOURCE: affinity <Object>

DESCRIPTION:
     If specified, the pod's scheduling constraints

     Affinity is a group of affinity scheduling rules.

FIELDS:
   Nodeaffinity < Object > # node affinity
     Describes node affinity scheduling rules for the pod.

   Podaffinity < Object > #pod affinity
     Describes pod affinity scheduling rules (e.g. co-locate this pod in the
     same node, zone, etc. as some other pod(s)).

   Podantiaffinity < Object > #pod anti affinity
     Describes pod anti-affinity scheduling rules (e.g. avoid putting this pod
     in the same node, zone, etc. as some other pod(s)).

 #Affinity and anti affinity are also divided into hard affinity and soft affinity. The node affinity mentioned in the previous section is not mentioned in detail
[[email protected] Scheduler]# kubectl explain pods.spec.affinity.podAffinity 

FIELDS:
   preferredDuringSchedulingIgnoredDuringExecution    <[]Object>
...

   requiredDuringSchedulingIgnoredDuringExecution    <[]Object>
...
#Pod anti affinity
[[email protected] Scheduler]# kubectl explain pods.spec.affinity.podAntiAffinity
FIELDS:
   preferredDuringSchedulingIgnoredDuringExecution    <[]Object>
...ffinityTerm; the
     node(s) with the highest sum are the most preferred.

   requiredDuringSchedulingIgnoredDuringExecution    <[]Object>
...

[[email protected] Scheduler]# kubectl explain pods.spec.affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution

   labelSelector    <Object>
     A label query over a set of resources, in this case pods.

   namespaces    <[]string>  
     namespaces specifies which namespaces the labelSelector applies to (matches
     against); null or empty list means "this pod's namespace"

   Topologykey < string > - required - # which tag is the location tag used by the topology tag (required field)
     This pod should be co-located (affinity) or not co-located (anti-affinity)
     with the pods matching the labelSelector in the specified namespaces, where
     co-located is defined as running on a node whose value of the label with
     key topologyKey matches that of any node on which any of the selected pods
     is running. Empty topologyKey is not allowed.

Example 1: podaffinity pod hard affinity

requiredDuringSchedulingIgnoredDuringExecution

[[email protected] Scheduler]# cat pod-affinity-required-demo.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis 
spec:
  Replicas: 1 #redis is a stateful application. Only 1 replica is run for testing
  selector:
    matchLabels:
      app: redis
      ctlr: redis
  template:
    metadata:
      labels:
        app: redis
        ctlr: redis
    spec:
      containers:
      - name: redis
        image: redis:6.0-alpine
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod-affinity-required
spec:
  replicas: 5
  selector:
    matchLabels:
      app: demoapp
      ctlr: pod-affinity-required
  template:
    metadata:
      labels:
        app: demoapp
        ctlr: pod-affinity-required
    spec:
      containers:
      - name: demoapp
        image: ikubernetes/demoapp:v1.0
      affinity:
        Podaffinity: #pod affinity
          Requiredduringschedulingignoredduringexecution: # hard affinity
          - labelSelector:
              matchExpressions:
              -{key: app, operator: in, values: ["redis"]} # complex indicates matching tags app = reids, CTLR = redis
              -{key: CTLR, operator: in, values: ["redis"]} # two expressions satisfy the and relationship at the same time
            Topologykey: rack # node label location label rack can be understood as satisfying the same relationship with labelselector on the same rack

[[email protected] Scheduler]# kubectl apply -f pod-affinity-required-demo.yaml 
deployment.apps/redis unchanged
deployment.apps/pod-affinity-required unchanged

[ [email protected] -Master scheduler]# kubectl get Pod - O wide # node is not labeled yet. Pod affinity hangs up
NAME                                     READY   STATUS    RESTARTS   AGE    IP              NODE        NOMINATED NODE   READINESS GATES
pod-affinity-required-5dd44f5d45-6vvhd   0/1     Pending   0          6m7s   <none>          <none>      <none>           <none>
pod-affinity-required-5dd44f5d45-c4hzl   0/1     Pending   0          6m7s   <none>          <none>      <none>           <none>
pod-affinity-required-5dd44f5d45-qm6zb   0/1     Pending   0          6m7s   <none>          <none>      <none>           <none>
pod-affinity-required-5dd44f5d45-t4mm5   0/1     Pending   0          6m7s   <none>          <none>      <none>           <none>
pod-affinity-required-5dd44f5d45-vs7dg   0/1     Pending   0          6m7s   <none>          <none>      <none>           <none>
redis-55f46d9795-r7pkz                   1/1     Running   0          10m    192.168.51.23   k8s-node3   <none>           <none>
  • Simulate different nodes in the same rack
[ [email protected] -Master scheduler]# kubectl label node k8s-node1 rack = foo # label node 1 rack
node/k8s-node1 labeled
[ [email protected] -Master scheduler]# kubectl label node k8s-node2 rack = bar # label node 2 rack
node/k8s-node2 labeled
[ [email protected] -Master scheduler]# kubectl label node k8s-node3 rack = Baz # label node 3 rack
node/k8s-node3 labeled
  • Hard affinity and redis run on the same node and both run on node3
[[email protected] Scheduler]# kubectl get pod  -o wide 
NAME                                     READY   STATUS    RESTARTS   AGE   IP              NODE        NOMINATED NODE   READINESS GATES
pod-affinity-required-5dd44f5d45-6vvhd   1/1     Running   0          21m   192.168.51.28   k8s-node3   <none>           <none>
pod-affinity-required-5dd44f5d45-c4hzl   1/1     Running   0          21m   192.168.51.26   k8s-node3   <none>           <none>
pod-affinity-required-5dd44f5d45-qm6zb   1/1     Running   0          21m   192.168.51.24   k8s-node3   <none>           <none>
pod-affinity-required-5dd44f5d45-t4mm5   1/1     Running   0          21m   192.168.51.25   k8s-node3   <none>           <none>
pod-affinity-required-5dd44f5d45-vs7dg   1/1     Running   0          21m   192.168.51.27   k8s-node3   <none>           <none>
redis-55f46d9795-r7pkz                   1/1     Running   0          25m   192.168.51.23   k8s-node3   <none>           <none>

Example 2: podaffinity pod soft affinity

preferredDuringSchedulingIgnoredDuringExecution

[[email protected] Scheduler]# cat pod-affinity-preferred-demo.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-preferred
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
      ctlr: redis-preferred
  template:
    metadata:
      labels:
        app: redis
        ctlr: redis-preferred
    spec:
      containers:
      - name: redis
        image: redis:6.0-alpine
        resources:
          requests:
            cpu: 200m
            memory: 512Mi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod-affinity-preferred
spec:
  replicas: 4
  selector:
    matchLabels:
      app: demoapp
      ctlr: pod-affinity-preferred
  template:
    metadata:
      labels:
        app: demoapp
        ctlr: pod-affinity-preferred
    spec:
      containers:
      - name: demoapp
        image: ikubernetes/demoapp:v1.0
        resources:
          requests:
            cpu: 500m
            memory: 100Mi
      affinity:
        podAffinity:
          Preferredduringschedulingignoredduringexecution: # soft affinity calculates the score with different weights and reenters the final affinity node
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - {key: app, operator: In, values: ["redis"]}     
                -{key: CTLR, operator: in, values: ["redis preferred"]} #pod includes the tags app = redis and CTLR = redis preferred, which are satisfied at the same time
              topologyKey: kubernetes. The IO / hostname # node location label takes different nodes as locations. The weight of the same node is 100, and the weight of all the same nodes is the highest
          - weight: 50
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - {key: app, operator: In, values: ["redis"]}
                - {key: ctlr, operator: In, values: ["redis-prefered"]}
              Topologykey: rack # node label position label is represented by rack, and the weight of rack is 50

[[email protected] Scheduler]# kubectl apply -f  pod-affinity-preferred-demo.yaml 

[ [email protected] -Master scheduler]# kubectl get Pod - O wide # runs on different nodes
NAME                                      READY   STATUS    RESTARTS   AGE    IP               NODE        NOMINATED NODE   READINESS GATES
pod-affinity-preferred-57d457968f-4lxvq   1/1     Running   0          5m6s   192.168.113.21   k8s-node1   <none>           <none>
pod-affinity-preferred-57d457968f-krb2f   1/1     Running   0          5m6s   192.168.113.20   k8s-node1   <none>           <none>
pod-affinity-preferred-57d457968f-mzckm   1/1     Running   0          5m6s   192.168.12.24    k8s-node2   <none>           <none>
pod-affinity-preferred-57d457968f-v8n8g   1/1     Running   0          5m6s   192.168.51.37    k8s-node3   <none>           <none>
redis-preferred-5d775df679-wtpgs          1/1     Running   0          5m6s   192.168.51.38    k8s-node3   <none>           <none>

Example 3: podantiaffinity hard anti affinity

requiredDuringSchedulingIgnoredDuringExecution

  • The four copies of deployment must run on different nodes, because there are only three node nodes, and all of them will have a pending internal exclusion
[[email protected] Scheduler]# cat pod-antiaffinity-required-demo.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod-antiaffinity-required
spec:
  Replicas: 4 # running 4 replicas
  selector:
    matchLabels:
      app: demoapp
      ctlr: pod-antiaffinity-required
  template:
    metadata:
      labels:
        app: demoapp
        ctlr: pod-antiaffinity-required
    spec:
      containers:
      - name: demoapp
        image: ikubernetes/demoapp:v1.0
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - {key: app, operator: In, values: ["demoapp"]}
              - key: ctlr
                operator: In
                Values: ["pod antiaffinity required"] #pod contains tags app = demoapp, CTLR = pod antiaffinity required, which are satisfied at the same time
            topologyKey: kubernetes. IO / hostname # takes the node as the location, indicating that each node can only run 1 pod

[[email protected] Scheduler]# kubectl apply -f  pod-antiaffinity-required-demo.yaml 

[[email protected] Scheduler]# kubectl get pod -o wide  
NAME                                         READY   STATUS    RESTARTS   AGE    IP               NODE        NOMINATED NODE   READINESS GATES
pod-antiaffinity-required-697f7d764d-hgkxj   1/1     Running   0          5m5s   192.168.113.34   k8s-node1   <none>           <none>
pod-antiaffinity-required-697f7d764d-n4zt9   1/1     Running   0          5m5s   192.168.12.34    k8s-node2   <none>           <none>
pod-antiaffinity-required-697f7d764d-psqfb   1/1     Running   0          5m5s   192.168.51.53    k8s-node3   <none>           <none>
Pod antiaffinity required 697f7d764d-q6t7m 0 / 1 pending 0 5m5s < none > < none > < none > < none > # suspended because there are only 3 node nodes