Understand kubernetes admission controller

Time:2022-5-16

Hello, I’m Zhang Jintao.

Articles published before meContainer image security in the cloud native EraIn (Series), I mentioned Kube apiserver, the core component of kubernetes cluster, which allows components from end users or clusters to communicate with it (for example, query, create, modify or delete kubernetes resources).

In this article, we will focus on a very important part of Kube apiserver request processing – admission controller

What is the access controller of k8s

Request processing flow in k8s

Before talking about what k8s admission controller is, let’s review the process of processing specific requests by kubernetes API.

Figure 1. The process of kubernetes API processing requests (from API handler to etcd persistence)

As shown in the figure above, the request of each API is received by Kube apiserver from the beginning and finally persisted to etcd, which is the request processing flow of kubernetes API.

It mainly includes the following parts:

  • API Handler–Mainly responsible for providing services and receiving requests.

For its internal implementation, the request comes firstFullHandlerChain(it is made byDefaultBuildHandlerChainBuilt) is adirectorobject

type director struct {
    name               string
    goRestfulContainer *restful.Container
    nonGoRestfulMux    *mux.PathRecorderMux
}

directorInitialize according to the configuration, ifgoRestfulContainerThe rootpath of WebServices is/apis, or if the request prefix matches the rootpath, enter the restful processing link.

  • Authentication–Certification process.

After the TLS connection is established, authentication processing will be carried out. If the request for authentication fails, the request will be rejected and 401 error code will be returned; If the authentication is successful, it will proceed to the authentication part. At present, there are many supported client authentication methods, such as x509 client certificate, bearer token, authentication based on user name and password, openid authentication, etc. Since these contents are not the focus of this article, we will skip them for the time being. Interested partners can leave a message in the comment area for discussion.

  • Authorization–Authentication process.

For kubernetes, it supports a variety of authentication modes, such as ABAC mode, RBAC mode and webhook mode. When creating a cluster, we can directly pass parameters to Kube apiserver for configuration, which will not be repeated here.

  • Mutating Admission–It refers to the access controller that can be used for change operation, which will be described in detail below.
  • Object Schema Validation–Schema verification of resource objects.
  • Validating Admission–Refers to the access controller that can be used to perform verification operations, which will be described in detail below.
  • ETCD–Etcd realizes the persistent storage of resources.

The above is a request processing process, in which changing permission and validating permission are our protagonists today. Let’s take a detailed look.

What is the admission controller

Access controller refers toAfter the request passes authentication and authorization, some codes or functions that can be used for change operation or verification operation.

The process of admission control is divided into two stages:

  • The first stage is to run the changing admission controller. It can modify the objects accepted by it, which leads to another role of it, changing the relevant resources as part of the request processing;
  • In the second stage, run the validating admission controller. It can only be verified and cannot modify any resource data;

It should be noted that some controllers can be both a change admission controller and a verification admission controller. If the admission controller of any stage rejects the request, the whole request will be rejected immediately and an error will be returned to the end user.

Why do I need admission controller

We mainly understand why we need access controller from two perspectives:

  • From the perspective of safety
    • We need to know whether the image source deployed in kubernetes cluster is trusted to avoid being attacked;
    • In general, try not to use the root user in the pod, or try not to open the privilege container;
  • From the perspective of governance
    • For example, to distinguish businesses / services through labels, you can check whether the corresponding label exists in the service through the admission controller;
    • For example, add resource quota restrictions to avoid oversold resources;

Access controller

Considering that these requirements are useful and indeed needed, kubernetes has implemented many built-in permission controllers. You can refer to the official documents for a detailed list:https://kubernetes.io/docs/re…

These built-in permission controllers are built together with Kube apiserver in the form of plug-ins. You can enable and shut them down. For example, the following parameters are used for control:

➜  bin ./kube-apiserver --help |grep admission-plugins    
      --admission-control strings              Admission is divided into two phases. In the first phase, only mutating admission plugins run. In the second phase, only validating admission plugins run. The names in the below list may represent a validating plugin, a mutating plugin, or both. The order of plugins in which they are passed to this flag does not matter. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. (DEPRECATED: Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.)
      --disable-admission-plugins strings      admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.
      --enable-admission-plugins strings       admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

Among so many admission controllers, there are two special ones, namelyMutatingAdmissionWebhookandValidatingAdmissionWebhook。 They do not really implement the corresponding policies, but provide an extensible way for Kube apiserver, which can be configured by usersMutatingAdmissionWebhookandValidatingAdmissionWebhookTo use self built services, and this method does not need to compile or restart Kube spiserver. It is completely dynamic and very convenient.

Let’s take a closer look.

Dynamic admission controller

Use the above mentionedMutatingAdmissionWebhookandValidatingAdmissionWebhookFor runtime configuration, the admission controller called in the form of webhook is the dynamic admission controller.

It is an HTTP callback mechanism used to receive and process admission requests, which is a web service. Currently, there are two types of admission webhooks:

  • validating admission webhook
  • mutating admission webhook

The mutating admission webhook will be called preferentially, and resources can be modified during this process.

If we need to ensure the final state of the object to perform some operations, we should consider using validating admission webhookBecause requests that reach this stage will not be modified.

Service conditions

  • Ensure that the kubernetes cluster version is at least v1.0 16 (for use)admissionregistration.k8s.io/v1API) or v1 9 (for easy use)admissionregistration.k8s.io/v1beta1 API);
  • Make sure it is enabledMutatingAdmissionWebhookandValidatingAdmissionWebhookAccess controller;
  • Make sure that is enabledadmissionregistration.k8s.io/v1beta1perhapsadmissionregistration.k8s.io/v1 API;

What is permission webhook

In fact, it is an ordinary HTTP server, which needs to be processedAdmissionReviewFor type resources, let’s take a look at an example, for exampleIngressResource access verification:

func (ia *IngressAdmission) HandleAdmission(obj runtime.Object) (runtime.Object, error) {

    review, isV1 := obj.(*admissionv1.AdmissionReview)
    if !isV1 {
        return nil, fmt.Errorf("request is not of type AdmissionReview v1")
    }

    if !apiequality.Semantic.DeepEqual(review.Request.Kind, ingressResource) {
        return nil, fmt.Errorf("rejecting admission review because the request does not contain an Ingress resource but %s with name %s in namespace %s",
            review.Request.Kind.String(), review.Request.Name, review.Request.Namespace)
    }

    status := &admissionv1.AdmissionResponse{}
    status.UID = review.Request.UID

    ingress := networking.Ingress{}

    codec := json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme, scheme, json.SerializerOptions{
        Pretty: true,
    })
    codec.Decode(review.Request.Object.Raw, nil, nil)
    _, _, err := codec.Decode(review.Request.Object.Raw, nil, &ingress)
    if err != nil {
        klog.ErrorS(err, "failed to decode ingress")
        status.Allowed = false
        status.Result = &metav1.Status{
            Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
            Message: err.Error(),
        }

        review.Response = status
        return review, nil
    }

    if err := ia.Checker.CheckIngress(&ingress); err != nil {
        klog.ErrorS(err, "invalid ingress configuration", "ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
        status.Allowed = false
        status.Result = &metav1.Status{
            Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
            Message: err.Error(),
        }

        review.Response = status
        return review, nil
    }

    klog.InfoS("successfully validated configuration, accepting", "ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
    status.Allowed = true
    review.Response = status

    return review, nil
}

The core processing logic is actually sent when processing the request webhookAdmissionReview, it will contain the resource objects to be verified. Then we verify or modify the resource object according to the actual needs.

Here are several points to note:

  • The processing of mutating webhook is serial, while validating webhook is parallel;
  • Although the processing of mutating webhook is serial, it does not guarantee the order;
  • Note that the processing of mutating webhook should be idempotent to avoid the result not meeting the expectations;
  • When processing requests, pay attention to handling all API versions of resource objects;

How to deploy administration webhook

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx-admission
webhooks:
  - name: validate.nginx.ingress.kubernetes.io
    matchPolicy: Equivalent
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    failurePolicy: Fail
    sideEffects: None
    admissionReviewVersions:
      - v1
    clientConfig:
      service:
        namespace: ingress-nginx
        name: ingress-nginx-controller-admission
        path: /networking/v1/ingresses

staywebhooksConfigure the specific connection information and trigger rules of webhook in.rulesYou can specify which specific actions are effective for which resources.

summary

This article mainly introduces the admission controller in kubernetes. By default, some have been compiled with Kube apiserver in the form of plug-in. In addition, we can also write our own dynamic admission controller to complete the relevant requirements.

Of course, at present, there are many ready-made tools in k8s ecology to help us complete these corresponding things. In many cases, we don’t need to develop corresponding services by ourselves. Later, I’ll share with you some mainstream tools that can be used for mutating and validating access control. Welcome to pay attention.


Welcome to subscribe to my article official account [moelove]