Tekton Pipelines–Task

Time:2020-2-11

Preface

Tekton pipelines is an open source implementation that configures and runs CI / CD style pipelines for your kubernetes application.

Pipelines create custom resources to declare pipelines as building blocks.

The custom resource is an extension of the kubernetes API. You can create custom kubernetes objects. After installing the custom resources, users can use kubectl to create and access their objects, just like the built-in resources such as pod and deployment. These resources run on a cluster and are implemented by kubernetes custom resource definition (CRD).

High level details about this design:

  • PipelinesDon’t know what triggers them, they can be created by events or manuallyPipelineRunTrigger.
  • TasksCan exist alone, and can be completely independent ofPipelinesCall. They are characterized by high cohesion and low coupling.
  • TasksCan depend on othersTasksCreated artifacts and parameters.
  • TasksCan beTaskRunsCall.
  • PipelineResourcesIs used asTasksInput and output parts.

Next, we will introduce the following building components one by one:

  • Task
  • TaskRun
  • Pipeline
  • PipelineRun
  • PipelineResource

Other reference topics unrelated to a specific component:

  • Labels
  • Logs

This article mainly introduces tasks.

Task

Task (or clustertask) is a collection of sequential steps that you want to run in a continuous integration process. The task will run within the pod on the cluster.

A task statement includes:

  • Inputs
  • Outputs
  • Steps

The scope of task is a namespace, while the scope of clustertask is the whole kubernetes cluster.

ClusterTask

Similar to task, but the scope is the entire cluster.

If you use clustertask, you should add the taskref type. The default type is task, which represents the task of the namespace.

apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: demo-pipeline
  namespace: default
spec:
  tasks:
    - name: build-skaffold-web
      taskRef:
        name: build-push
        kind: ClusterTask
      params: ....

The function of ask is the same as that of clustertask, so all references to task below also describe clustertask

grammar

To define a profile for a task resource, you can specify the following fields:

  • Must write:

    • apiVersion-Specify the API version, for exampletekton.dev/v1alpha1.
    • kind– designateTaskResource object
    • metadata-Specify data to uniquely identifyTaskResource objects, such asname.
    • spec– forTaskThe resource object specifies configuration information. Task steps must be defined by any of the following fields:

      • steps-Specifies one or more container images to run in the task.
  • Optional:

    • inputs– specify youTaskParameters andPipelineResources
    • outputs– specify youTaskProducedPipelineResources
    • volumes-Specify one or more to use forTaskinstepsThe mount volume of
    • stepTemplate– designatecontainerStep definition to be used as allstepFoundation. .
    • sidecars-Specifies that the sidercar container runs with steps

The following example is an invalid example where most of the possible configuration fields are used:

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: example-task-name
spec:
  inputs:
    resources:
      - name: workspace
        type: git
    params:
      - name: pathToDockerFile
        type: string
        description: The path to the dockerfile to build
        default: /workspace/workspace/Dockerfile
  outputs:
    resources:
      - name: builtImage
        type: image
  steps:
    - name: ubuntu-example
      image: ubuntu
      args: ["ubuntu-build-example", "SECRETS-example.md"]
    - image: gcr.io/example-builders/build-example
      command: ["echo"]
      args: ["$(inputs.params.pathToDockerFile)"]
    - name: dockerfile-pushexample
      image: gcr.io/example-builders/push-example
      args: ["push", "$(outputs.resources.builtImage.url)"]
      volumeMounts:
        - name: docker-socket-example
          mountPath: /var/run/docker.sock
  volumes:
    - name: example-volume
      emptyDir: {}

Steps

Steps field is required. You define one or more step fields to define the task body.

If multiple steps are defined, when tasks are called by taskrun, they are executed in the defined order.

Each step in the task must specify a container image that complies with the container specification. For each step field or container image you define:

  • Run and evaluate the container image in order of the configuration file.
  • Container mirrors ran until they completed or detected the first failure.
  • If the container image does not have the largest resource request in all container images in the task, the CPU, memory, and temporary storage resource requests are set to zero. This ensures that the pod executing the task requests only the resources needed to perform any single container image in the task, rather than the sum of all container image resource requests.

Inputs

A task can declare its required inputs, which can be one or all of the following:

  • parameter
  • Input resources

parameter

A task can declare the input parameters that must be provided to the task during a task run. Some sample use cases for this include:

  • You need to know what compilation flags a task uses when building an application.
  • You need to know how to name tasks that have built artifacts.

Parameter names are limited to alphanumeric characters – and ‘,’ and can only begin with alphabetic characters and ”. For example, foois bar UU is a valid parameter name, barisba $or 0banana is not.

Each declared parameter has a type field, which is assumed to be a string if not provided by the user. Another possible type is an array – useful, for example, when you need to provide a dynamic number of compile flags to a task building an application. When the actual parameter value is provided, the type it resolves is validated against the type field.

usage

The following example shows how to parameterize tasks and pass these parameters from taskrun to task.
Input parameters in the form of $(inputs. Params. Foo) are replaced in the step (see also variable replacement).
The following task declares an input parameter named “flags” and uses it in the steps.args list.

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: task-with-parameters
spec:
  inputs:
    params:
      - name: flags
        type: array
      - name: someURL
        type: string
  steps:
    - name: build
      image: my-builder
      args: ["build", "$(inputs.params.flags)", "url=$(inputs.params.someURL)"]

The following taskrun provides a dynamic number of strings in the flags parameter:

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: run-with-parameters
spec:
  taskRef:
    name: task-with-parameters
  inputs:
    params:
      - name: flags
        value: 
          - "--set"
          - "arg1=foo"
          - "--randomflag"
          - "--someotherflag"
      - name: someURL
        value: "http://google.com"

Input resources

Use the pipelineresources field you enter to provide your task with the data or context it needs.

Outputs

Task definitions can include input and output pipelineresource declarations. If only a specific resource set is declared in the output, a copy of the resource that is expected to be uploaded or shared in the next task is located under path / workspace / output / resource_name /.

resources:
  outputs:
    name: storage-gcs
    type: gcs
steps:
  - image: objectuser/run-java-jar #https://hub.docker.com/r/objectuser/run-java-jar/
    command: [jar]
    args:
      ["-cvf", "-o", "/workspace/output/storage-gcs/", "projectname.war", "*"]
    env:
      - name: "FOO"
        value: "world"

Note: if the task depends on the output resource function, the container in the task step field cannot load anything in path / workspace / output.

In the following example, tasktar-artifactThe resource is used as both input and output, so the input resource is downloaded to the directory customworkspace (specified in targetpath). StepuntarExtract the tar file into the tar scratch space directory,edit-tarAdd a new file, last steptar-it-upCreate a new tar file and place it in the / workspace / customworkspace / directory. After performing the task step, the (New) tar file in the directory / workspace / customworkspace will be uploaded to thetar-artifactIn the bucket defined in the resource definition

resources:
  inputs:
    name: tar-artifact
    targetPath: customworkspace
  outputs:
    name: tar-artifact
steps:
 - name: untar
    image: ubuntu
    command: ["/bin/bash"]
    args: ['-c', 'mkdir -p /workspace/tar-scratch-space/ && tar -xvf /workspace/customworkspace/rules_docker-master.tar -C /workspace/tar-scratch-space/']
 - name: edit-tar
    image: ubuntu
    command: ["/bin/bash"]
    args: ['-c', 'echo crazy > /workspace/tar-scratch-space/rules_docker-master/crazy.txt']
 - name: tar-it-up
   image: ubuntu
   command: ["/bin/bash"]
   args: ['-c', 'cd /workspace/tar-scratch-space/ && tar -cvf /workspace/customworkspace/rules_docker-master.tar rules_docker-master']

Volumes

Specify one or more volumes for your task or all steps.

For example, use volumes to do one of the following common tasks:

  • Mount k8s secret.
  • Create an emptydir volume to act as a cache for use in multiple build steps. Consider using persistent volumes for build caching.
  • Mount k8s configmap
  • Mount the docker socket of the host to build the container image using the dockerfile. Note: using docker build on cluster to build container image is very insecure. Please use kaniko instead. This is for demonstration purposes only

Step Template

Specifies a container configuration that will be used as the basis for all steps in the task. The configuration in a single step overrides or merges with the configuration of the step template.

In the following example, the task specifies a steptemplate with the environment variable foo set to bar. The first step uses foo for that value, but in the second step, you override Foo and set it to Baz.

stepTemplate:
  env:
    - name: "FOO"
      value: "bar"
steps:
  - image: ubuntu
    command: [echo]
    args: ["FOO is ${FOO}"]
  - image: ubuntu
    command: [echo]
    args: ["FOO is ${FOO}"]
    env:
      - name: "FOO"
        value: "baz"

Sidecars

Specifies the list of containers to run with steps. These containers can provide auxiliary functions, such as docker in docker or running simulation API server, to make your application hit during test.
Sidecar starts before executing the task step and destroys after all steps are completed. For more information about the sidecar lifecycle, see the [taskrun documentation]
(https://github.com/tektoncd/p…

In the following example, you run docker in docker’s sidecar so that it can be used in one step to build a docker image

steps:
  - image: docker
    name: client
    workingDir: /workspace
    command:
      - /bin/sh
      - -c
      - |
        cat > Dockerfile << EOF
        FROM ubuntu
        RUN apt-get update
        ENTRYPOINT ["echo", "hello"]
        EOF
        docker build -t hello . && docker run hello
        docker images
    volumeMounts:
      - mountPath: /var/run/
        name: dind-socket
sidecars:
  - image: docker:18.05-dind
    name: server
    securityContext:
      privileged: true
    volumeMounts:
      - mountPath: /var/lib/docker
        name: dind-storage
      - mountPath: /var/run/
        name: dind-socket
volumes:
  - name: dind-storage
    emptyDir: {}
  - name: dind-socket
    emptyDir: {}

Variable Substitution

Tasks support replacing strings with values from all inputs and outputs.

The following variable substitution syntax can be used to reference input parameters in the task specification, where < name > is the name of the parameter:

$(inputs.params.<name>)

Parameter values from resources can also be accessed using variable substitution.

Variable substitution with parameter type of array

Reference parameters of type array are extended to insert array elements at the location of the reference string.

Therefore, the following parameters are used:

inputs:
    params:
      - name: array-param
        value: 
          - "some"
          - "array"
          - "elements"

Then command: [“first”, “$(inputs. Params. Array param)”, “last”] will become command: [“first”, “some”, “array”, “elements”, “last”]

Note that array parameters must be referenced in a fully isolated string in a large string array. Any other attempt to reference an array is invalid and will raise an error.

For example, if build args is a declared parameter of type array, this is an invalid step because the string is not isolated:

- name: build-step
      image: gcr.io/cloud-builders/some-image
      args: ["build", "additionalArg $(inputs.params.build-args)"]

Similarly, references to build args in non array fields are not valid:

- name: build-step
      image: "$(inputs.params.build-args)"
      args: ["build", "args"]

A valid reference to the build args parameter is isolated and in the qualified field (args in this case):

- name: build-step
      image: gcr.io/cloud-builders/some-image
      args: ["build", "$(inputs.params.build-args)", "additonalArg"]

Variable Substitution within Volumes

You can set the task volume name and different types of volumes. Current support includes widely used volume types, such as configmap, secret, and persistent volume claim. This is about how to use this example in the task definition.

Examples

For example, a task that encapsulates a dockerfile build might look like this:

Note: using docker build on cluster to build container image is very insecure. Please use kaniko instead. This is for demonstration purposes only.

spec:
  inputs:
    resources:
      - name: workspace
        type: git
    params:
      # These may be overridden, but provide sensible defaults.
      - name: directory
        type: string
        description: The directory containing the build context.
        default: /workspace
      - name: dockerfileName
        type: string
        description: The name of the Dockerfile
        default: Dockerfile
  outputs:
    resources:
      - name: builtImage
        type: image
  steps:
    - name: dockerfile-build
      image: gcr.io/cloud-builders/docker
      workingDir: "$(inputs.params.directory)"
      args:
        [
          "build",
          "--no-cache",
          "--tag",
          "$(outputs.resources.image)",
          "--file",
          "$(inputs.params.dockerfileName)",
          ".",
        ]
      volumeMounts:
        - name: docker-socket
          mountPath: /var/run/docker.sock

    - name: dockerfile-push
      image: gcr.io/cloud-builders/docker
      args: ["push", "$(outputs.resources.image)"]
      volumeMounts:
        - name: docker-socket
          mountPath: /var/run/docker.sock

  # As an implementation detail, this Task mounts the host's daemon socket.
  volumes:
    - name: docker-socket
      hostPath:
        path: /var/run/docker.sock
        type: Socket

Use an external volume

Mount multiple volumes:

spec:
  steps:
    - image: ubuntu
      entrypoint: ["bash"]
      args: ["-c", "curl https://foo.com > /var/my-volume"]
      volumeMounts:
        - name: my-volume
          mountPath: /var/my-volume

    - image: ubuntu
      args: ["cat", "/etc/my-volume"]
      volumeMounts:
        - name: my-volume
          mountPath: /etc/my-volume

  volumes:
    - name: my-volume
      emptyDir: {}
      

Use kubernetes configmap as volume

spec:
 inputs:
   params:
     - name: CFGNAME
       type: string
       description: Name of config map
     - name: volumeName
       type: string
       description: Name of volume
 steps:
   - image: ubuntu
     entrypoint: ["bash"]
     args: ["-c", "cat /var/configmap/test"]
     volumeMounts:
       - name: "$(inputs.params.volumeName)"
         mountPath: /var/configmap

 volumes:
   - name: "$(inputs.params.volumeName)"
     configMap:
       name: "$(inputs.params.CFGNAME)"

Using secret as the environment variable

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
 name: goreleaser
spec:
 inputs:
   params:
   - name: package
     type: string
     description: base package to build in
   - name: github-token-secret
     type: string
     description: name of the secret holding the github-token
     default: github-token
   resources:
   - name: source
     type: git
     targetPath: src/$(inputs.params.package)
 steps:
 - name: release
   image: goreleaser/goreleaser
   workingdir: /workspace/src/$(inputs.params.package)
   command:
   - goreleaser
   args:
   - release
   env:
   - name: GOPATH
     value: /workspace
   - name: GITHUB_TOKEN
     valueFrom:
       secretKeyRef:
         name: $(inputs.params.github-token-secret)
         key: bot-token

Recommended Today

Golang naming convention

All naming of function name, variable name, constant name, type name, statement label and package name in go language follow a simple naming rule. It must start with a letter or underscore (), followed by any number of letters, numbers or underscores. In the go language, upper and lower case letters are different. There are […]