Héctor Valls: Software Development Blog

Cover Image for Running multiple Kubernetes jobs with completion index

Running multiple Kubernetes jobs with completion index

If you need to run a Kubernetes job many times, use .spec.completions. Job will be completed when run successfully .spec.completions times. If you need to differentiate between the runs, set .spec.completionMode to Indexed. That will inject the environment variable JOB_COMPLETION_INDEX with the index of the run.

For example, in the past, I used this to deploy multiple customers of my application. The customers to be deployed are listed in a ConfigMap and the Job used the JOB_COMPLETION_INDEX to select the customer. Something like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: customers-list
data:
  customers: "customerA,customerB,customerC,customerD,customerE,customerF,customerG,customerH,customerI,customerJ"

---

apiVersion: batch/v1
kind: Job
metadata:
  name: customer-deployer
spec:
  completions: 10 # Customers list length
  completionMode: Indexed
  template:
    spec:
      restartPolicy: Never
      containers:
      - name: customer-deployer
        image: customer-deployer:1.0
        command: ["sh", "-c"]
        args:
        - |
          CUSTOMER_NAME=$(echo $CUSTOMER_LIST | cut -d',' -f $((JOB_COMPLETION_INDEX + 1)))
          /app/deploy-customer.sh $CUSTOMER_NAME
        env:
        - name: CUSTOMER_LIST
          valueFrom:
            configMapKeyRef:
              name: customers-list
              key: customers

You can use .spec.parallelism to set the maximum number of parallel runs.