Using #BuildKit for Cloud Native Build in #GitLab

After my talk about BuildKit at DockerCon Live 2020 I wanted to provide a detailed answer to a question from the audience. I was asked how to use BuildKit in GitLab CI and this post will explain this for running the BuildKit daemon as a service and using BuildKit daemonless in a job.

When the question came up during the talk, I only answered that I have successfully tested running BuildKit in GitLab CI but the situaton did not allow for a detailed answer so I will present you with two answers to make up for leaving this open. Both will be using rootless to reduce the attack surface against the container runtime.

Answer 1: Running the BuildKit daemon as a service

The most obvious approach to running BuildKit in GitLab CI is executing the daemon and the CLI separately. As buildkitd must be running to execute buildctl, a service can be defined to achieve this. Services are launched before executing any commands of the job and will not be stopped until after the job has finished. The approach is based on exposing the BuildKit daemon on TCP. There are a few things to note in the following example:

• Using an alias for the service makes accessing it easier from the pipeline job

• buildctl accepts an environment variable BUILDKIT_HOST how to access the daemon

• The image moby/buildkit:rootless starts the BuildKit daemon by default, therefore the entrypoint must be overridden

stages:
- build

buildkitd:
  stage: build

  services:
  - alias: buildkitd
    name: moby/buildkit:rootless
    command:
    - "--oci-worker-no-process-sandbox"
    - "--addr"
    - "tcp://0.0.0.0:1234"
  variables:
    BUILDKIT_HOST: tcp://buildkitd:1234

  image:
    name: moby/buildkit:rootless
    entrypoint: [ "sh", "-c" ]
  script:
  - |
    buildctl build \
        --frontend=dockerfile.v0 \
        --local context=. \
        --local dockerfile=.
  tags:
  - docker

Answer 2: Running BuildKit daemonless

Instead of running the BuildKit daemon as a service, it is possible to use the buildctl-daemonless.sh script to transparently start the daemon in the background and then launch buildctl with the specified parameters - this is called daemonless. The advantage is that the daemon is not exposed on the network and will be only accessible inside the pipeline job. There are a few things to note in the following example:

• The daemonless script accepts an environment variable BUILDKITD_FLAGS with parameters for the BuildKit daemon

• The image moby/buildkit:rootless starts the BuildKit daemon by default therefore the entrypoint must be overridden

stages:
- build

daemonless:
  stage: build
  image:
    name: moby/buildkit:rootless
    entrypoint: [ "sh", "-c" ]
  variables:
    BUILDKITD_FLAGS: --oci-worker-no-process-sandbox
  script:
  - |
    buildctl-daemonless.sh build \
        --frontend=dockerfile.v0 \
        --local context=. \
        --local dockerfile=.
  tags:
  - docker

Sidenote 1: Using rootless with GitLab.com vs. self-hosted

The above pipeline jobs will work out-of-the-box on shared runners provided by GitLab.com. When using the same on self-hosted runners, make sure that seccomp profile and apparmor profile are set to unconfined.

The only alternative is to abandon using rootless and abandon to decrease the attach surface against the container runtime.

Sidenote 2: Parameter --oci-worker-no-process-sandbox

The BuildKit repository provides a detailed explanation why the parameter --oci-worker-no-process-sandbox is required when using rootless.