Application management in k8s — creation and maintenance of private helm chart


In the previous article, I introduced:

  • How to use gitlab CI for continuous deployment.
  • How to deploy and apply helm and helmfile to kubernetes cluster.

But the key link is missing: create a chart of your project, so that we can deploy our project to the cluster through helm. This article will show you how to create and maintain chart, so as to get through the whole process from code submission to deployment.

Where’s chart

Before creating a chart, the first thing to consider is where to put the chart. At present, the common practice in the community is to store the chart separately from the application code, put the chart in a separate warehouse, and the CI is responsible for packaging and updating. The advantages are:

  • Chart’s release cycle does not have to be synchronized with the application.
  • Commit tree is clearer.

Because charts of open source projects may be installed in other places or become a dependency of a chart, you must publish these charts. For closed source business projects like ours, chart will not be relied on by external projects. I prefer to put chart directly in the application code warehouse. The reasons are as follows:

  • Ci packaging does not have to be configured separately.
  • The release rhythm keeps pace with the application, and it is convenient to roll back when there is a problem.

For example, the charts of several main projects of our company are all stored in their respective code warehousesdeploy/chartUnder the table of contents:

Application management in k8s -- creation and maintenance of private helm chart

Creating chart and Helmfile.yaml

After determining the storage location, you can start to create the chart. The specific operation is not difficult, just execute the following command:

cd deploy
Helm create API # API is the name of the chart you want to create

However, it should be noted that by default, helm will create a directory namedapiThe subdirectory of (depending on the name you fill in) is used to store charts. So you need to do it manuallymv api chart. Don’t try to run it directlyhelm create chartBecause in the process of creating a new chart, the name you fill in will be used in chart templates, which is more difficult to replace.

In addition, we createdhelmfile.yamlIn CI, we just need to runhelmfile -e $ENV applyThat’s it.

  - name: stable
  - name: bitnami

      - ./deploy/values/env.stg.yaml.gotmpl
      - ./deploy/values/env.prd.yaml.gotmpl

  wait: true
  timeout: 180

  - name: {{ .Environment.Values.releaseName }}
    namespace: {{ .Environment.Values.namespace }}
    Chart:. / deploy / chart # directly fill in the relative path of chart
      - ./deploy/env/secrets.{{ .Environment.Values.envName }}.yaml
      - ./deploy/values/values.{{ .Environment.Name }}.yaml.gotmpl

For the basic use of helmfile, please refer to my previous blogApplication management under k8s — understanding helmfile

It is worth mentioning that we have configured two default parameters of helm.

Among them,waitConfigure astrueIn this way, when running in the CI environment, the pipeline will not change to the successful state until the deployed resource state changes to ready. If this parameter is not enabled, as long as the resource is created normally, helm will complete and exit. At this time, pod may not be started, unable to schedule, failed to pull the image, etc. these problems should belong to the category of deployment failure. Therefore, in order to make the execution result of CI more accurate, we turn on this parameter.

In addition, in order to prevent too long pipeline time,timeoutSet to180Seconds.

Add or adjust resource template

The newly created chart contains some templates of common resources, such as progress, deployment, etc. You can add other resources as needed. For template syntax, please refer to the official go template document.

If you don’t know how to create new resources, I recommend this…This includes many common kubernetes resource examples, which you can directly copy and transform to go template. There are two points that need to be modified. Take configmap as an example

apiVersion: v1
kind: ConfigMap
  name: {{ include "api.fullname" . }}
    {{- include "api.labels" . | nindent 4 }}
# ...

First, change the resource name to the same as other resources, and use the template rendering fullname. Secondly, it should be revisedlabelsGenerally speaking, you can use the labels rendered by helm’s own auxiliary function.

be careful,api.fullnameandapi.labelsIt’s not fixed,apiIt’s your chart name.

Edit values

After adding other resources, you can refer to more custom values, for example:

apiVersion: v1
kind: ConfigMap
# ...
  APP_ENV: {{ }}

We willAPP_ENVConfigured as in valuesapp.envThe value of. After quoting the new values, I recommend modifying the contents of chartvalues.yamlTo keep the template and values synchronized, so that when adding a new environment, there will be a “basic” values for reference. For example:

# ...
  env: production
# ...

Rely on other charts

Usually, our applications rely on some well-known open source components, such as redis, mysql, etc. Helm allows us to declare other charts as dependencies. During the installation process of chart, we will also install the dependent charts, so as to achieve the purpose of installing applications and related services at the same time.

The new version of helm V3 provides support for chart v2dependencies.yamlIt’s abandoned. editChart.yaml, adddependenciesPart of it is enough

apiVersion: v2
# ...
  -Name: redis # chart name
    Version: 10.5.10 # chart version
    repository: # Chart Repo

implementhelm dep buildYou can pull the dependency and generate itChart.lock. Finally, because the distribution package of the dependent chart is stored in thechartsSubdirectories, so it is recommended thatdeploy/chart/chartsAdd path to.gitignoreFile to prevent dependent code from being submitted to git version management.


It is not easy to create and maintain a reliable chart. In the process of continuous improvement, we put a lot of energy into improvement and optimization. If you are interested, please read the best practice document of helm chart provided by helm and bitnam. I believe it will be helpful to you

This work adoptsCC agreementReprint must indicate the author and the link of this article

Former WinForm and PHP engineer. Now prefer Golang and Rust, and mainly working on DevSecOps and Kubernetes.

Recommended Today

Third party calls wechat payment interface

Step one: preparation 1. Wechat payment interface can only be called if the developer qualification has been authenticated on wechat open platform, so the first thing is to authenticate. It’s very simple, but wechat will charge 300 yuan for audit 2. Set payment directory Login wechat payment merchant platform( pay.weixin.qq . com) — > Product […]