Kubebuilder practice 5: operator coding

Time:2022-5-10

Welcome to my GitHub

https://github.com/zq2599/blog_demos

Content: classification and summary of all original articles and supporting source code, involving Java, docker, kubernetes, Devops, etc;

Links to series articles

  1. One of kubebuilder’s actual combat: preparation
  2. Kubebuilder practice 2: experience kubebuilder for the first time
  3. Kubebuilder practice 3: a quick overview of basic knowledge
  4. Kube builder practice 4: operator requirements description and design
  5. Kubebuilder practice 5: operator coding
  6. Kubebuilder practice 6: build, deploy and run
  7. Kubebuilder practical battle 7: webhook
  8. Kubebuilder practice 8: Notes on knowledge points

Overview of this article

  • This is the fifth in the series of “Kube builder actual combat”. All the previous efforts (environment preparation, knowledge reserve, demand analysis, data structure and business logic design) are to realize the previous design with coding;
  • Now that we are fully prepared, we don’t need much words now. Let’s start!

Source download

  • The complete source code in this actual combat can be downloaded from GitHub. The address and link information are shown in the table below(https://github.com/zq2599/blog_demos):
name link remarks
Project Home https://github.com/zq2599/blog_demos The project is on the home page of GitHub
Git warehouse address (HTTPS) https://github.com/zq2599/blog_demos.git The warehouse address of the source code of the project, HTTPS protocol
Git warehouse address (SSH) [email protected]:zq2599/blog_demos.git The warehouse address of the source code of the project, SSH protocol
  • There are multiple folders in this git project. Kubebuilder related applications arekubebuilderFolder, as shown in the red box below:

在这里插入图片描述

  • There are several subfolders under kubebuilder folder. The corresponding source code of this article is inelasticwebDirectory, as shown in the red box below:

在这里插入图片描述

New project elasticweb

  • New namedelasticwebTo create a folder namedelasticwebProject with domain ofcom.bolingcavalry
go mod init elasticweb
kubebuilder init --domain com.bolingcavalry
  • Then CRD, execute the following command to create relevant resources:
kubebuilder create api \
--group elasticweb \
--version v1 \
--kind ElasticWeb
  • Then open the whole project with IDE. Here is Golan:

在这里插入图片描述

CRD code

  • Open fileapi/v1/elasticweb_types.go, make the following changes:
  1. Modify the data structure elasticwebspec and add four fields designed above;
  2. Modify the data structure elasticwebstatus and add a field designed above;
  3. Add the string method to make it easy for us to view when printing the log. Note that the realqps field is a pointer, so it may be null and needs to be judged null;
  • completeelasticweb_types.goAs follows:
package v1

import (
	"fmt"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"strconv"
)

//Expected state
type ElasticWebSpec struct {
	//The image corresponding to the business service, including the name tag
	Image string `json:"image"`
	//The host port occupied by the service. External requests access the service of the pod through this port
	Port *int32 `json:"port"`

	//QPS upper limit of a single pod
	SinglePodQPS *int32 `json:"singlePodQPS"`
	//Current total QPS of the whole business
	TotalQPS *int32 `json:"totalQPS"`
}

//In the actual state, the values in the data structure are calculated by the business code
type ElasticWebStatus struct {
	//Total QPS actually supported in current kubernetes
	RealQPS *int32 `json:"realQPS"`
}

// +kubebuilder:object:root=true

// ElasticWeb is the Schema for the elasticwebs API
type ElasticWeb struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   ElasticWebSpec   `json:"spec,omitempty"`
	Status ElasticWebStatus `json:"status,omitempty"`
}

func (in *ElasticWeb) String() string {
	var realQPS string

	if nil == in.Status.RealQPS {
		realQPS = "nil"
	} else {
		realQPS = strconv.Itoa(int(*(in.Status.RealQPS)))
	}

	return fmt.Sprintf("Image [%s], Port [%d], SinglePodQPS [%d], TotalQPS [%d], RealQPS [%s]",
		in.Spec.Image,
		*(in.Spec.Port),
		*(in.Spec.SinglePodQPS),
		*(in.Spec.TotalQPS),
		realQPS)
}

// +kubebuilder:object:root=true

// ElasticWebList contains a list of ElasticWeb
type ElasticWebList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []ElasticWeb `json:"items"`
}

func init() {
	SchemeBuilder.Register(&ElasticWeb{}, &ElasticWebList{})
}
  • Execute in elasticweb directorymake installCRD can be deployed to kubernetes:
[email protected] elasticweb % make install
/Users/zhaoqin/go/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
kustomize build config/crd | kubectl apply -f -
Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/elasticwebs.elasticweb.com.bolingcavalry created
  • After successful deployment, useapi-versionsThe GV can be found by the command:

在这里插入图片描述

Review business logic

  • After the design and coding of the core data structure, it’s time to write the business logic code. You still remember the business process designed above. A brief review is as follows:
    在这里插入图片描述

  • Open fileelasticweb_controller.goNext, let’s add content gradually;

Add resource access

  • Our elasticweb will query, add, modify and other operations on service and deployment resources. Therefore, the operation permissions of these resources are required. Add two lines of comments in the red box below, so that the code generation tool will add corresponding permissions in RBAC configuration:

在这里插入图片描述

Constant definition

  • First prepare the constant. It can be seen that the CPU and memory used by each pod are fixed here. You can also define it in spec instead, so it can be imported from the outside. In addition, only 0.1 CPU is allocated for each pod, mainly because I am poor and can’t afford a good CPU. You can adjust the value as appropriate:
const (
	//App tag name in deployment
	APP_NAME = "elastic-app"
	//Port number of Tomcat container
	CONTAINER_PORT = 8080
	//CPU resource request for a single pod
	CPU_REQUEST = "100m"
	//Maximum CPU resources of a single pod
	CPU_LIMIT = "100m"
	//Memory resource application of a single pod
	MEM_REQUEST = "512Mi"
	//Maximum memory resources of a single pod
	MEM_LIMIT = "512Mi"
)

Method getexpectreplicas

  • There is a very important logic: calculate how many pods are needed according to the QPS and total QPS of a single pod. Let’s encapsulate this logic into a method for use:
/Calculate pod quantity according to single QPS and total QPS
func getExpectReplicas(elasticWeb *elasticwebv1.ElasticWeb) int32 {
	//QPS of single pod
	singlePodQPS := *(elasticWeb.Spec.SinglePodQPS)

	//Expected total QPS
	totalQPS := *(elasticWeb.Spec.TotalQPS)

	//Replicas is the number of replicas to create
	replicas := totalQPS / singlePodQPS

	if totalQPS%singlePodQPS > 0 {
		replicas++
	}

	return replicas
}

Method createserviceifnotexists

  • Encapsulating the operation of creating a service into a method makes the logic of the main code clearer and more readable;
  • There are several points to note when creating a service:
  1. Check whether the service exists before creating it;
  2. Associate the service with the CRD instance elasticweb (controllerutil. Setcontrollerreference method), so that when elasticweb is deleted, the service will be automatically deleted without our intervention;
  3. It is used when creating a serviceclient-goTools, it is recommended that you read “client go actual combat series”. The more proficient the tools are, the more enjoyable the coding will be;
  • The complete method of creating a service is as follows:
//New service
func createServiceIfNotExists(ctx context.Context, r *ElasticWebReconciler, elasticWeb *elasticwebv1.ElasticWeb, req ctrl.Request) error {
	log := r.Log.WithValues("func", "createService")

	service := &corev1.Service{}

	err := r.Get(ctx, req.NamespacedName, service)

	//If there is no error in the query result and the service is proved to be normal, no operation will be done
	if err == nil {
		log.Info("service exists")
		return nil
	}

	//If the error is not notfound, an error is returned
	if !errors.IsNotFound(err) {
		log.Error(err, "query service error")
		return err
	}

	//Instantiate a data structure
	service = &corev1.Service{
		ObjectMeta: metav1.ObjectMeta{
			Namespace: elasticWeb.Namespace,
			Name:      elasticWeb.Name,
		},
		Spec: corev1.ServiceSpec{
			Ports: []corev1.ServicePort{{
				Name:     "http",
				Port:     8080,
				NodePort: *elasticWeb.Spec.Port,
			},
			},
			Selector: map[string]string{
				"app": APP_NAME,
			},
			Type: corev1.ServiceTypeNodePort,
		},
	}

	//This step is very critical!
	//After the association is established, the deployment will be deleted when the elastic web resource is deleted
	log.Info("set reference")
	if err := controllerutil.SetControllerReference(elasticWeb, service, r.Scheme); err != nil {
		log.Error(err, "SetControllerReference error")
		return err
	}

	//Create service
	log.Info("start create service")
	if err := r.Create(ctx, service); err != nil {
		log.Error(err, "create service error")
		return err
	}

	log.Info("create service success")

	return nil
}

Method createdeployment

  • Encapsulating the operation of creating deployment in one method is also to keep the trunk logic concise;
  • There are also several points to note in the method of creating deployment:
  1. Call getexpectreplicas method to get the number of pods to be created, which is an important parameter when creating deployment;
  2. The CPU and memory resources required by each pod are also parameters of deployment;
  3. Associate the deployment with elastic web, so that the deployment will be automatically deleted when the elastic web is deleted;
  4. Similarly, the client-go client tool is used to create deployment resources;
//New deployment
func createDeployment(ctx context.Context, r *ElasticWebReconciler, elasticWeb *elasticwebv1.ElasticWeb) error {
	log := r.Log.WithValues("func", "createDeployment")

	//Calculate the expected number of pods
	expectReplicas := getExpectReplicas(elasticWeb)

	log.Info(fmt.Sprintf("expectReplicas [%d]", expectReplicas))

	//Instantiate a data structure
	deployment := &appsv1.Deployment{
		ObjectMeta: metav1.ObjectMeta{
			Namespace: elasticWeb.Namespace,
			Name:      elasticWeb.Name,
		},
		Spec: appsv1.DeploymentSpec{
			//The number of copies is calculated
			Replicas: pointer.Int32Ptr(expectReplicas),
			Selector: &metav1.LabelSelector{
				MatchLabels: map[string]string{
					"app": APP_NAME,
				},
			},

			Template: corev1.PodTemplateSpec{
				ObjectMeta: metav1.ObjectMeta{
					Labels: map[string]string{
						"app": APP_NAME,
					},
				},
				Spec: corev1.PodSpec{
					Containers: []corev1.Container{
						{
							Name: APP_NAME,
							//Use the specified image
							Image:           elasticWeb.Spec.Image,
							ImagePullPolicy: "IfNotPresent",
							Ports: []corev1.ContainerPort{
								{
									Name:          "http",
									Protocol:      corev1.ProtocolSCTP,
									ContainerPort: CONTAINER_PORT,
								},
							},
							Resources: corev1.ResourceRequirements{
								Requests: corev1.ResourceList{
									"cpu":    resource.MustParse(CPU_REQUEST),
									"memory": resource.MustParse(MEM_REQUEST),
								},
								Limits: corev1.ResourceList{
									"cpu":    resource.MustParse(CPU_LIMIT),
									"memory": resource.MustParse(MEM_LIMIT),
								},
							},
						},
					},
				},
			},
		},
	}

	//This step is very critical!
	//After the association is established, the deployment will be deleted when the elastic web resource is deleted
	log.Info("set reference")
	if err := controllerutil.SetControllerReference(elasticWeb, deployment, r.Scheme); err != nil {
		log.Error(err, "SetControllerReference error")
		return err
	}

	//Create deployment
	log.Info("start create deployment")
	if err := r.Create(ctx, deployment); err != nil {
		log.Error(err, "create deployment error")
		return err
	}

	log.Info("create deployment success")

	return nil
}

Method updatestatus

  • Whether you create a deployment resource object or adjust the number of pods of an existing deployment, you need to modify the status after these operations are completed, that is, the actual status, so that the external can know how much QPS is supported by the current elasticweb anytime and anywhere. Therefore, you need to package the operation of modifying the status into one method and use it for multiple scenarios. The calculation logic of status is very simple: the number of pods multiplied by the QPS of each pod is the total QPS, The code is as follows:
//Update the latest status after completing the processing of pod
func updateStatus(ctx context.Context, r *ElasticWebReconciler, elasticWeb *elasticwebv1.ElasticWeb) error {
	log := r.Log.WithValues("func", "updateStatus")

	//QPS of single pod
	singlePodQPS := *(elasticWeb.Spec.SinglePodQPS)

	//Total pod
	replicas := getExpectReplicas(elasticWeb)

	//After the pod is created, the actual QPS of the current system: QPS of a single pod * total number of pods
	//If the field has not been initialized, initialize it first
	if nil == elasticWeb.Status.RealQPS {
		elasticWeb.Status.RealQPS = new(int32)
	}

	*(elasticWeb.Status.RealQPS) = singlePodQPS * replicas

	log.Info(fmt.Sprintf("singlePodQPS [%d], replicas [%d], realQPS[%d]", singlePodQPS, replicas, *(elasticWeb.Status.RealQPS)))

	if err := r.Update(ctx, elasticWeb); err != nil {
		log.Error(err, "update instance error")
		return err
	}

	return nil
}

Trunk code

  • After the above details have been handled, the main process can be started. With the assignment of the previous flow chart, the code of the main process can be written easily, as shown below. Enough comments have been added and will not be repeated:
func (r *ElasticWebReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) {
	//Context will be used
	ctx := context.Background()
	log := r.Log.WithValues("elasticweb", req.NamespacedName)

	// your logic here

	log.Info("1. start reconcile logic")

	//Instantiate data structure
	instance := &elasticwebv1.ElasticWeb{}

	//Query through the client tool. The query criteria are
	err := r.Get(ctx, req.NamespacedName, instance)

	if err != nil {

		//If there is no instance, an empty result will be returned, so that the reconcile method will not be called immediately
		if errors.IsNotFound(err) {
			log.Info("2.1. instance not found, maybe removed")
			return reconcile.Result{}, nil
		}

		log.Error(err, "2.2 error")
		//Return error message to external
		return ctrl.Result{}, err
	}

	log.Info("3. instance : " + instance.String())

	//Find deployment
	deployment := &appsv1.Deployment{}

	//Query with client tools
	err = r.Get(ctx, req.NamespacedName, deployment)

	//An exception occurs when searching, and the processing logic of finding no result
	if err != nil {
		//If there is no instance, it will be created
		if errors.IsNotFound(err) {
			log.Info("4. deployment not exists")

			//If there is no demand for QPS and there is no deployment at this time, nothing will be done
			if *(instance.Spec.TotalQPS) < 1 {
				log.Info("5.1 not need deployment")
				//Return
				return ctrl.Result{}, nil
			}

			//First create a service
			if err = createServiceIfNotExists(ctx, r, instance, req); err != nil {
				log.Error(err, "5.2 error")
				//Return error message to external
				return ctrl.Result{}, err
			}

			//Create deployment now
			if err = createDeployment(ctx, r, instance); err != nil {
				log.Error(err, "5.3 error")
				//Return error message to external
				return ctrl.Result{}, err
			}

			//If the creation is successful, the status is updated
			if err = updateStatus(ctx, r, instance); err != nil {
				log.Error(err, "5.4. error")
				//Return error message to external
				return ctrl.Result{}, err
			}

			//Once created successfully, you can return
			return ctrl.Result{}, nil
		} else {
			log.Error(err, "7. error")
			//Return error message to external
			return ctrl.Result{}, err
		}
	}

	//If the deployment is found and no error is returned, follow the logic below

	//Calculate the expected number of copies according to the single QPS and the total QPS
	expectReplicas := getExpectReplicas(instance)

	//Expected number of copies of current deployment
	realReplicas := *deployment.Spec.Replicas

	log.Info(fmt.Sprintf("9. expectReplicas [%d], realReplicas [%d]", expectReplicas, realReplicas))

	//If equal, it returns directly
	if expectReplicas == realReplicas {
		log.Info("10. return now")
		return ctrl.Result{}, nil
	}

	//If not, adjust it
	*(deployment.Spec.Replicas) = expectReplicas

	log.Info("11. update deployment's Replicas")
	//Update deployment through client
	if err = r.Update(ctx, deployment); err != nil {
		log.Error(err, "12. update deployment replicas error")
		//Return error message to external
		return ctrl.Result{}, err
	}

	log.Info("13. update status")

	//If the replication of deployment is updated successfully, the status will be updated
	if err = updateStatus(ctx, r, instance); err != nil {
		log.Error(err, "14. update status error")
		//Return error message to external
		return ctrl.Result{}, err
	}

	return ctrl.Result{}, nil
}
  • So far, the whole elastic web operator coding has been completed. Limited to space, let’s put the deployment, operation, image making and other operations in the next article;

You’re not alone. Xinchen’s original accompanies you all the way

  1. Java series
  2. Spring series
  3. Docker series
  4. Kubernetes series
  5. Database + middleware series
  6. Devops series

Welcome to official account: programmer Xinchen

Wechat search “programmer Xinchen”. I’m Xinchen. I look forward to traveling with you in the Java World
https://github.com/zq2599/blog_demos

Recommended Today

Mybatis plus multi table associated query column ‘ID’ in where claim is approximate reprint

Reprinted from https://blog.csdn.net/qq_3383… 1、 Error reporting informationCaused by: Column ‘xxxx’ in where clause is ambiguous 2、 Error reporting reasonBoth table person and table class have field ID and name, so alias should be added to distinguish them. PersonVOMapper.java public interface PersonVOMapper extends BaseMapper<PersonVO> { List<PersonVO> getPersonVOList(@Param(Constants.WRAPPER) Wrapper<PersonVO> queryWrapper); } PersonVOMapper.xml <?xml version=”1.0″ encoding=”UTF-8″?> <!DOCTYPE mapper […]