Writing a #Kubernetes controller in #bash to handle DNS for #traefik

I am using traefik as the ingress controller in multiple Kubernetes clusters. Those instances are running standalone (without the integrated high availability). To make sure that users are able to reach traefik, a DNS record points to the host IP. So far, an init container was responsible for updating DNS if the traefik pod restarts. But recently I decided to decouple the DNS update from traefik. This led to writing a Kubernetes controller to watch the traefik pod for restarts and update DNS accordingly. This post provides details about writing the controller in bash.

Receiving Kubernetes events

Events can be displayed on the command line using kubectl:

kubectl get pods --selector ${LABEL_SELECTOR} --watch --output-watch-events

When adding --output json these events are shown in JSON and can be processed by script logic:

kubectl get pods --selector ${LABEL_SELECTOR} --watch --output-watch-events --output json | while read EVENT; do
    # my code
done

The events contains a type in .type as well as the object in .object:

{
  "type": "ADDED",
  "object": {}
}

Events can have one of the types ADDED, MODIFIED and DELETED.

When a service like traefik is restarted, the pod is removed and a new pod is created. Kubernetes produces multiple events to follow the state of the old as well as the new object as they are terminated and scheduled.

Note: I do not know why but jq is not able to parse the JSON provided by kubectl unless I remove managed fields (.metadata.managedFields):

kubectl get pods --selector ${LABEL_SELECTOR} --watch --output-watch-events --output json | \
    jq --compact-output --monochrome-output --unbuffered 'del(.object.metadata.managedFields)' | \
    while read EVENT; do
        # my code
done

traefik-dns

I have published my code for traefik-dns on GitHub. It follows pods in a specific namespace and using a label selector and relies on external-dns to update DNS.

It works in the following way:

  1. It follows a specific pod specified by namespace and label selector
  2. The controller runs in an endless loop to process events
  3. When the controller starts it receives an event of the type ADDED representing the current state (when the pod exists)
  4. The control loop processes events to follow the termination of the pod
  5. When a new pod is scheduled the events are also followed
  6. As soon as the new pod is running, the control loop calls a handler to update a DNSEndpoint object

The project is in just a proof of concept. Do not use it in production.

Alternative: shell-operator

After finishing the first version of traefik-dns I came across shell-operator. This allows to focus on the task at hand instead of implementing the control loop myself. I will take a closer look at it and update traefik-dns to use shell-operator.

Feedback is always welcome! If you'd like to get in touch with me concerning the contents of this article, please use Twitter.