In this Kubernetes lab, we will explore Kubernetes Jobs and CronJobs.

Unlike Kubernetes Deployments, Kubernetes Jobs are designed to quit after they have accomplished the task (successful or not). Kubernetes CronJobs are the jobs that are repeated according to a schedule pattern Linux administrators know from crontab.

Step 0: Access the Kubernetes Playground

As always, we access the Katacode Kubernetes Playground to perform our tests. Just click on launch.sh on the left pane, and we are ready to go.

Phase 1: Jobs

Step 1.1: Create a Hello World Job

We will create a hello world job, which says „hello world“, wait for 5 seconds and then quit with a „bye“. For that, we will create a YAML file with a Kubernetes object of kind batch/job:

apiVersion: batch/v1
kind: Job
metadata:
  name: hello-world-job
spec:
  template:
    spec:
      containers:
      - name: hello-world-container
        image: centos:7
        command: ["bash"]
        args:
        - "-c"
        - "echo 'Hello World'; sleep 5; echo 'Bye'"
      restartPolicy: Never

Step 1.2: Create and watch the Job PODs

Now let us create the job and observe the created PODs with a watch command:

kubectl apply -f job.yaml

# output: job.batch/hello-world-job created
watch 'kubectl get pod'

# output every some seconds:
NAME                    READY   STATUS              RESTARTS   AGE
hello-world-job-zrsg9   0/1     ContainerCreating   0          4s

NAME                    READY   STATUS              RESTARTS   AGE
hello-world-job-zrsg9   0/1     ContainerCreating   0          7s

NAME                    READY   STATUS    RESTARTS   AGE
hello-world-job-zrsg9   1/1     Running   0          9s

NAME                    READY   STATUS    RESTARTS   AGE
hello-world-job-zrsg9   1/1     Running   0          10s

NAME                    READY   STATUS    RESTARTS   AGE
hello-world-job-zrsg9   1/1     Running   0          11s

NAME                    READY   STATUS      RESTARTS   AGE
hello-world-job-zrsg9   0/1     Completed   0          13s

NAME                    READY   STATUS      RESTARTS   AGE
hello-world-job-zrsg9   0/1     Completed   0          14s

NAME                    READY   STATUS      RESTARTS   AGE
hello-world-job-zrsg9   0/1     Completed   0          15s

The watch command can be finished by pressing <Ctrl> – c.

Step 1.3 (optional): Get Job Status

Let us view the number of completions of the job: the hello-world-job has completed one out or one runs:

kubectl get job
NAME              COMPLETIONS   DURATION   AGE
hello-world-job   1/1           13s        5m52s

We can see more information using a describe command. Among others, we will see, how many PODs are running, how many have succeeded and how many have failed: 0 Running / 1 Succeeded / 0 Failed

kubectl describe jobs

# output:
Name:           hello-world-job
Namespace:      default
Selector:       controller-uid=efdd427d-b154-11e9-8bd4-0242ac110011
Labels:         controller-uid=efdd427d-b154-11e9-8bd4-0242ac110011
                job-name=hello-world-job
Annotations:    kubectl.kubernetes.io/last-applied-configuration:
                  {"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{},"name":"hello-world-job","namespace":"default"},"spec":{"template":{"sp...
Parallelism:    1
Completions:    1
Start Time:     Sun, 28 Jul 2019 16:29:57 +0000
Completed At:   Sun, 28 Jul 2019 16:30:10 +0000
Duration:       13s
Pods Statuses:  0 Running / 1 Succeeded / 0 Failed
Pod Template:
  Labels:  controller-uid=efdd427d-b154-11e9-8bd4-0242ac110011
           job-name=hello-world-job
  Containers:
   hello-world-container:
    Image:      centos:7
    Port:       
    Host Port:  
    Command:
      bash
    Args:
      -c
      echo 'Hello World'; sleep 5; echo 'Bye'
    Environment:  
    Mounts:       
  Volumes:        
Events:
  Type    Reason            Age    From            Message
  ----    ------            ----   ----            -------
  Normal  SuccessfulCreate  8m17s  job-controller  Created pod: hello-world-job-zrsg9

Step 1.4: Get Job Output

As expected, the POD has sent greetings to STDOUT:

kubectl logs hello-world-job-zrsg9

# output:
Hello World
Bye

Note: the POD name will  be different in your case.

Step 1.5: Review Job Parameters

Now let us explore and tweak some parameters. For that, we can review the YAML output of the job:

kubectl get -o yaml job hello-world-job

# output:
apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{},"name":"hello-world-job","namespace":"default"},"spec":{"template":{"spec":{"containers":[{"args":["-c","echo 'Hello World'; sleep 5; echo 'Bye'"],"command":["bash"],"image":"centos:7","name":"hello-world-container"}],"restartPolicy":"Never"}}}}
  creationTimestamp: "2019-07-28T16:29:57Z"
  labels:
    controller-uid: efdd427d-b154-11e9-8bd4-0242ac110011
    job-name: hello-world-job
  name: hello-world-job
  namespace: default
  resourceVersion: "5712"
  selfLink: /apis/batch/v1/namespaces/default/jobs/hello-world-job
  uid: efdd427d-b154-11e9-8bd4-0242ac110011
spec:
  backoffLimit: 6
  completions: 1
  parallelism: 1
  selector:
    matchLabels:
      controller-uid: efdd427d-b154-11e9-8bd4-0242ac110011
  template:
    metadata:
      creationTimestamp: null
      labels:
        controller-uid: efdd427d-b154-11e9-8bd4-0242ac110011
        job-name: hello-world-job
    spec:
      containers:
      - args:
        - -c
        - echo 'Hello World'; sleep 5; echo 'Bye'
        command:
        - bash
        image: centos:7
        imagePullPolicy: IfNotPresent
        name: hello-world-container
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Never
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  completionTime: "2019-07-28T16:30:10Z"
  conditions:
  - lastProbeTime: "2019-07-28T16:30:10Z"
    lastTransitionTime: "2019-07-28T16:30:10Z"
    status: "True"
    type: Complete
  startTime: "2019-07-28T16:29:57Z"
  succeeded: 1

Step 1.6: Increase the Number of Completions

From the output above, we can deduce that we can increase the number of completions by adding a completions directive. Let us add it to the job.yaml file accordingly:

apiVersion: batch/v1
kind: Job
metadata:
  name: hello-world-job
spec:
  completions: 5 
  template:
    spec:
      containers:
      - name: hello-world-container
        image: centos:7
        command: ["bash"]
        args:
        - "-c"
        - "echo 'Hello World'; sleep 5; echo 'Bye'"
      restartPolicy: Never

Now let us try to apply the change:

kubectl apply -f job.yaml

# output:
The Job "hello-world-job" is invalid: spec.completions: Invalid value: 5: field is immutable

This did not work, because the field is immutable. Instead, we need to delete the job and create a new job,  based on the new YAML file:

kubectl delete job hello-world-job
# output: job.batch "hello-world-job" deleted

kubectl apply -f job.yaml
# output: job.batch/hello-world-job created

Now, we can see that the job has been performed 5 times:

kubectl get jobs

# output:
NAME              COMPLETIONS   DURATION   AGE
hello-world-job   5/5           36s        103s

Each job has been performed in its own POD:

kubectl get pods

# output:
NAME                    READY   STATUS      RESTARTS   AGE
hello-world-job-4n25s   0/1     Completed   0          2m20s
hello-world-job-6lkmx   0/1     Completed   0          2m41s
hello-world-job-jznwv   0/1     Completed   0          2m48s
hello-world-job-sqghk   0/1     Completed   0          2m27s
hello-world-job-tl6nl   0/1     Completed   0          2m34s

Step 1.7: Parallel Execution of Jobs

From the values in the AGE column, we can deduce that the PODs have run sequentially. Now, let us try to run four of the five PODs in parallel. For that, we add the blue line:

apiVersion: batch/v1
kind: Job
metadata:
  name: hello-world-job
spec:
  completions: 5
  parallelism: 4
  template:
    spec:
      containers:
      - name: hello-world-container
        image: centos:7
        command: ["bash"]
        args:
        - "-c"
        - "echo 'Hello World'; sleep 5; echo 'Bye'"
      restartPolicy: Never

We apply the configuration by deleting and re-creating the job:

kubectl delete job hello-world-job
# output:job.batch "hello-world-job" deleted

kubectl apply -f job.yaml
# output: job.batch/hello-world-job created

kubectl get pods
# output:
NAME                    READY   STATUS      RESTARTS   AGE
hello-world-job-6zbtz   0/1     Completed   0          9s
hello-world-job-b7vwd   0/1     Completed   0          9s
hello-world-job-rphkn   1/1     Running     0          3s  <----- late comer
hello-world-job-sd7z5   0/1     Completed   0          9s
hello-world-job-w4k6j   0/1     Completed   0          9s

kubectl get pods
# output:
NAME                    READY   STATUS      RESTARTS   AGE
hello-world-job-6zbtz   0/1     Completed   0          18s
hello-world-job-b7vwd   0/1     Completed   0          18s
hello-world-job-rphkn   0/1     Completed   0          12s  <----- late comer
hello-world-job-sd7z5   0/1     Completed   0          18s
hello-world-job-w4k6j   0/1     Completed   0          18s

Indeed, we see above that four of the five PODs were executed in parallel, while the fifth POD has waited for a free slot.

Step 1.8: Configure an Execution Timeout

Now, let us consider a situation, where a job takes too long to make any sense. For such situations, we can define an execution timeout called „activeDeadlineSeconds“:

# job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: hello-world-job
spec:
  completions: 5
  parallelism: 2
  activeDeadlineSeconds: 15
  template:
    spec:
      containers:
      - name: hello-world-container
        image: centos:7
        command: ["bash"]
        args:
        - "-c"
        - "echo 'Hello World'; sleep 5; echo 'Bye'"
      restartPolicy: Never

We set the number of completions to five with may two parallel PODs, while we allow for the execution time of 15 seconds.

kubectl get pods

# output:
NAME                    READY   STATUS        RESTARTS   AGE
hello-world-job-2rktd   0/1     Terminating   0          17s
hello-world-job-5hhbf   0/1     Completed     0          29s
hello-world-job-7qnj9   0/1     Completed     0          23s
hello-world-job-9d555   0/1     Completed     0          29s
hello-world-job-k4j7t   0/1     Completed     0          23s

After some time, the Terminating POD just disappears:

kubectl get pods

# output:
NAME                    READY   STATUS      RESTARTS   AGE
hello-world-job-5hhbf   0/1     Completed   0          34s
hello-world-job-7qnj9   0/1     Completed   0          28s
hello-world-job-9d555   0/1     Completed   0          34s
hello-world-job-k4j7t   0/1     Completed   0          28s

Let us review the events globally:

kubectl get events

# output 
<output omitted>
7m59s       Normal    SuccessfulCreate   job/hello-world-job         Created pod: hello-world-job-9d555
7m59s       Normal    SuccessfulCreate   job/hello-world-job         Created pod: hello-world-job-5hhbf
7m53s       Normal    SuccessfulCreate   job/hello-world-job         Created pod: hello-world-job-k4j7t
7m53s       Normal    SuccessfulCreate   job/hello-world-job         Created pod: hello-world-job-7qnj9
7m47s       Normal    SuccessfulCreate   job/hello-world-job         Created pod: hello-world-job-2rktd
7m44s       Normal    SuccessfulDelete   job/hello-world-job         Deleted pod: hello-world-job-2rktd
7m44s       Warning   DeadlineExceeded   job/hello-world-job         Job was active longer than specified deadline

Similar information can be seen on the kubectl describe job command.

Phase 2: CronJobs

In this phase, we will perform changes to our job.yaml file, so the job will be executed periodically.

Step 2.1: Create a CronJob YAML File

First. let us copy the job file. You can cut&paste it from here. However, I have created it from the original job.yaml file in order to better see the differences:

cp job.yaml cronjob.yaml

# cronjob.yaml
apiVersion: batch/v1beta1         # <---------- batch/v1 -> batch/v1beta1
kind: CronJob                     # <---------- Job -> CronJob
metadata:
  name: hello-world-cronjob       # <---------- new name
spec:
  schedule: "*/2 * * * *"         # <---------- UNIX style cron tab
  jobTemplate:                    # <---------- JobTemplate
    spec:                         # <---------- JobTemplate Spec
      template:                   # <---------- from here:
                                  # <---------- same as Job spec, but indented for 4 spaces
        spec:
          containers:
          - name: hello-world-container
            image: centos:7
            command: ["bash"]
            args:
            - "-c"
            - "echo 'Hello World'; sleep 5; echo 'Bye'"
          restartPolicy: Never

We can see that a CronJob has a UNIX-style crontab string and a JobTemplate, which is a template to create jobs from it. Therefore, it is not surprising to find the job specs within the template again.

Step 2.2: Create a CronJob from a YAML File

Now, let us create the resource:

kubectl apply -f cronjob.yaml
# output: cronjob.batch/hello-world-cronjob created

Step 2.3: Check the Status

We check the resource:

kubectl get cronjobs

# output:
NAME                  SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
hello-world-cronjob   */2 * * * *   False     0        <none>          6s

It has not yet run. Therefore, no job and no POD has been created yet:

kubectl get jobs
# output: No resources found.

kubectl get pods
# output: No resources found.

Let us wait for 2 minutes and try again (several times):

k get pods
# output:
NAME                                   READY   STATUS              RESTARTS   AGE
hello-world-cronjob-1564340520-jljqw   0/1     ContainerCreating   0          4s

k get pods
# output:
NAME                                   READY   STATUS    RESTARTS   AGE
hello-world-cronjob-1564340520-jljqw   1/1     Running   0          9s

k get pods
# output:
NAME                                   READY   STATUS    RESTARTS   AGE
hello-world-cronjob-1564340520-jljqw   1/1     Running   0          13s

k get pods
# output:
NAME                                   READY   STATUS      RESTARTS   AGE
hello-world-cronjob-1564340520-jljqw   0/1     Completed   0          1m47s

Something similar can be seen on the Job level:

kubectl get jobs
# output:
NAME                             COMPLETIONS   DURATION   AGE
hello-world-cronjob-1564340640   1/1           7s         4m34s
hello-world-cronjob-1564340760   1/1           7s         2m34s
hello-world-cronjob-1564340880   1/1           7s         34s

Now let us check the status of the cronjob again:

kubectl get cronjobs

# output:
NAME                  SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
hello-world-cronjob   */2 * * * *   False     0        19s             10m

We can see that the last job was scheduled 19 sec ago.

As a summary, a CronJob is an object that creates Jobs periodically by a schedule defined in the UNIX-style crontab.

Previous

CKA Labs (6) — Kubernetes API

Next

Planned: Kubernetes Deployments, Replicasets and DaemonSets; see:

CKA Labs (1) — The Certified Kubernetes Administrator Challenge