In this lab, we will explore the Kubernetes API. Firstly, we will run curl commands against the API to read and to create PODs. This requires us extracting keys and certificates from the kube config file. Secondly, we will show how to explore the API resources by finding and reviewing API cache files.

References:

Step 1: Accessing the API with cURL via SSL

Per default, the Kubernetes API is secured with SSL. Under the hood, the kubectl are performing SSL calls to the API as well. However, can we access the API via simple curl commands? Yes, we can, if we retrieve the appropriate certificates and keys from the Kubernetes configuration file.

Step 1.1 (optional): Retrieve the Client Certificate

Note: Step 1.1 to 1.4 are optional, since 1.5 will combile all necessary data retrieval steps into a for-loop.

We can retrieve the client certificate with a single grep command:

CLIENT_CERT=$(grep client-cert ~/.kube/config | awk '{print $2}')

We can review the certificate with an echo command:

echo $CLIENT_CERT

# output:
LS0tLS1CRUdJT...
< output omitted >
NBVEUtLS0tLQo=

Step 1.2 (optional): Retrieve the Client Key

With a similar command, we can retrieve the

CLIENT_KEY_DATA=$(grep client-key-data ~/.kube/config | awk '{print $2}')

Let us review the content of the variable again:

echo $CLIENT_KEY_DATA

# output:
ROeDJWRVVsVndwT...
< output omitted >
BSU0EgUFJJVkFURSBLRVktLS0tLQo=

Step 1.3 (optional): Retrieve the Certificate Authority Data

Same again, but now for the Certificate Authority Data:

CERTIFICATE_AUTHORITY_DATA=$(grep certificate-authority-data ~/.kube/config | awk '{print $2}')
echo $CERTIFICATE_AUTHORITY_DATA

# output:
LS0tLS1CRUdJ...
< output omitted >
...VElGSUNBVEUtLS0tLQo=

Step 1.4 (optional): Write Data to PEM Files

Now we have written all necessary data into environment variables, we can create PEM files from them as follows. The data we have retrieved is base 64 encoded, but we need the original data, so we need to decode the data as follows:

echo $CLIENT_CERT                | base64 -d > client-cert.pem
echo $CLIENT_KEY_DATA            | base64 -d > client-key-data.pem
echo $CERTIFICATE_AUTHORITY_DATA | base64 -d > certificate-authority-data.pem

Step 1.5 (all-in-one alternative): Retrieve Data from config File

This step is combining steps 1.1 to 1.4 in a simple for-loop. We just retrieve the data from the config file and directly write it to the PEM files:

for item in client-cert client-key-data certificate-authority-data; 
do 
  echo $item; 
  grep $item ~/.kube/config | awk '{print $2}' | base64 -d > $item.pem; 
done

Step 1.6: Get API URL

Let us now retrieve the API URL from the configuration file

API_URL=$(kubectl config view | grep server | awk '{print $2}')
echo $API_URL

# output:
https://172.17.0.14:6443

Step 1.7: Perform API Calls via curl

Step 1.7.1: GET: Read Data

Now those three variables can be used to perform an API call. Here, we will see a list of resources the API is offering to us:

curl -s --cert ./client-cert.pem \
        --key ./client-key-data.pem \
        --cacert ./certificate-authority-data.pem $API_URL/api/v1 | grep \"name\"

# output:
      "name": "bindings",
      "name": "componentstatuses",
      "name": "configmaps",
      "name": "endpoints",
      "name": "events",
      "name": "limitranges",
      "name": "namespaces",
      "name": "namespaces/finalize",
      "name": "namespaces/status",
      "name": "nodes",
      "name": "nodes/proxy",
      "name": "nodes/status",
      "name": "persistentvolumeclaims",
      "name": "persistentvolumeclaims/status",
      "name": "persistentvolumes",
      "name": "persistentvolumes/status",
      "name": "pods",                 <------------- we will use this one
      "name": "pods/attach",
      "name": "pods/binding",
      "name": "pods/eviction",
      "name": "pods/exec",
      "name": "pods/log",
      "name": "pods/portforward",
      "name": "pods/proxy",
      "name": "pods/status",
      "name": "podtemplates",
      "name": "replicationcontrollers",
      "name": "replicationcontrollers/scale",
      "name": "replicationcontrollers/status",
      "name": "resourcequotas",
      "name": "resourcequotas/status",
      "name": "secrets",
      "name": "serviceaccounts",
      "name": "services",
      "name": "services/proxy",
      "name": "services/status",

Above, the API has provided us a list of API resources. Let us explore one of them: the „pods“ resource. The output consists of many lines. Therefore, we look at the first 40 lines of the output only:

curl -s --cert ./client-cert.pem  \
        --key ./client-key-data.pem \
        --cacert ./certificate-authority-data.pem \
        $API_URL/api/v1/pods | head -n 40

# output:
{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/pods",
    "resourceVersion": "9467"
  },
  "items": [
    {
      "metadata": {
        "name": "coredns-fb8b8dccf-67mbh",
        "generateName": "coredns-fb8b8dccf-",
        "namespace": "kube-system",
        "selfLink": "/api/v1/namespaces/kube-system/pods/coredns-fb8b8dccf-67mbh",
        "uid": "326628cc-adbc-11e9-b657-0242ac110015",
        "resourceVersion": "477",
        "creationTimestamp": "2019-07-24T02:39:02Z",
        "labels": {
          "k8s-app": "kube-dns",
          "pod-template-hash": "fb8b8dccf"
        },
        "ownerReferences": [
          {
            "apiVersion": "apps/v1",
            "kind": "ReplicaSet",
            "name": "coredns-fb8b8dccf",
            "uid": "325d3fb9-adbc-11e9-b657-0242ac110015",
            "controller": true,
            "blockOwnerDeletion": true
          }
        ]
      },
      "spec": {
        "volumes": [
          {
            "name": "config-volume",
            "configMap": {
              "name": "coredns",
              "items": [
                {

The output is organized as follows:

{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": { ... },
  "items": [
    { ...POD Info... },
    { ...POD Info... },

      ...

    { ...POD Info... }
   ]
}

Inside the curly brackets, we will find (rather lengthy) POD information, which we will explore in more detail after we have created our own POD.

Step 1.7.2: POST: Create an Object

We also can create a POD using a curl command by using the -X POST option:

curl -s -X POST \
     --cert ./client-cert.pem \
     --key ./client-key-data.pem \
     --cacert ./certificate-authority-data.pem \
     $API_URL/api/v1/namespaces/default/pods \
     -H 'Content-Type: application/json' \
-d'
{
    "kind": "Pod",
    "apiVersion": "v1",
    "metadata":{
        "name": "curlpod",
        "namespace": "default",
        "labels": {
            "name": "examplepod"
        }
    },
    "spec": {
        "containers": [{
            "name": "nginx",
            "image": "nginx",
            "ports": [{"containerPort": 80}]
        }]
    }
}
'

The output of that command contains the „selfLink“ that can be used to re-read the data from API any time later:

{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "curlpod",
    "namespace": "default",
    "selfLink": "/api/v1/namespaces/default/pods/curlpod",
    "uid": "1681e87c-adce-11e9-b657-0242ac110015",
    "resourceVersion": "10756",
    "creationTimestamp": "2019-07-24T04:47:06Z",
    "labels": {
      "name": "examplepod"
    }
  },
  "spec": {
    "volumes": [
      {
        "name": "default-token-rmj9w",
        "secret": {
          "secretName": "default-token-rmj9w",
          "defaultMode": 420
        }
      }
    ],
    "containers": [
      {
        "name": "nginx",
        "image": "nginx",
        "ports": [
          {
            "containerPort": 80,
            "protocol": "TCP"
          }
        ],
        "resources": {

        },
        "volumeMounts": [
          {
            "name": "default-token-rmj9w",
            "readOnly": true,
            "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
          }
        ],
        "terminationMessagePath": "/dev/termination-log",
        "terminationMessagePolicy": "File",
        "imagePullPolicy": "Always"
      }
    ],
    "restartPolicy": "Always",
    "terminationGracePeriodSeconds": 30,
    "dnsPolicy": "ClusterFirst",
    "serviceAccountName": "default",
    "serviceAccount": "default",
    "securityContext": {

    },
    "schedulerName": "default-scheduler",
    "tolerations": [
      {
        "key": "node.kubernetes.io/not-ready",
        "operator": "Exists",
        "effect": "NoExecute",
        "tolerationSeconds": 300
      },
      {
        "key": "node.kubernetes.io/unreachable",
        "operator": "Exists",
        "effect": "NoExecute",
        "tolerationSeconds": 300
      }
    ],
    "priority": 0,
    "enableServiceLinks": true
  },
  "status": {
    "phase": "Pending",
    "qosClass": "BestEffort"
  }
}

After that, a POD should be up and running in the default namespace:

kubectl get pods

# output:
NAME      READY   STATUS    RESTARTS   AGE
curlpod   1/1     Running   0          28s

Step 1.7.3: SelfLink

Cool. Now let us explore the „selfLink“: „/api/v1/namespaces/default/pods/curlpod“:

curl -s --cert ./client-cert.pem  \
        --key ./client-key-data.pem \
        --cacert ./certificate-authority-data.pem \
        $API_URL/api/v1/namespaces/default/pods/curlpod

# output:
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "curlpod",
< output omitted >
<same as output of the curl -X POST command above>

Step 1.7.4: PodList revisited

Interestingly, the PodList we had retrieved above does not contain an exact copy of that output: „kind“ and „apiVersion“ are missing:

curl -s --cert ./client-cert.pem \
        --key ./client-key-data.pem \
        --cacert ./certificate-authority-data.pem \
        $API_URL/api/v1/pods | head -n 150

# output:
{
  "kind": "PodList",    <---------------- implicit: kind=Pod of all items below
  "apiVersion": "v1",   <---------------- implicit: apiVersion=v1 for all items
  "metadata": {
    "selfLink": "/api/v1/pods",
    "resourceVersion": "1357"
  },
  "items": [
    {
                      <------------------ "kind" and "apiVersion" are missing here
      "metadata": {
        "name": "curlpod",
        "namespace": "default",
        "selfLink": "/api/v1/namespaces/default/pods/curlpod",
        "uid": "555e44ed-add1-11e9-a7b7-0242ac110019",
        "resourceVersion": "1260",
        "creationTimestamp": "2019-07-24T05:10:20Z",
        "labels": {
          "name": "examplepod"
        }
      },
      "spec": {
        "volumes": [
          {
            "name": "default-token-f4qdl",
            "secret": {
              "secretName": "default-token-f4qdl",
              "defaultMode": 420
            }
          }
        ],
        "containers": [
          {
            "name": "nginx",
            "image": "nginx",
            "ports": [
              {
                "containerPort": 80,
                "protocol": "TCP"
              }
            ],
            "resources": {

            },
            "volumeMounts": [
              {
                "name": "default-token-f4qdl",
                "readOnly": true,
                "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount"
              }
            ],
            "terminationMessagePath": "/dev/termination-log",
            "terminationMessagePolicy": "File",
            "imagePullPolicy": "Always"
          }
        ],
        "restartPolicy": "Always",
        "terminationGracePeriodSeconds": 30,
        "dnsPolicy": "ClusterFirst",
        "serviceAccountName": "default",
        "serviceAccount": "default",
        "nodeName": "node01",
        "securityContext": {

        },
        "schedulerName": "default-scheduler",
        "tolerations": [
          {
            "key": "node.kubernetes.io/not-ready",
            "operator": "Exists",
            "effect": "NoExecute",
            "tolerationSeconds": 300
          },
          {
            "key": "node.kubernetes.io/unreachable",
            "operator": "Exists",
            "effect": "NoExecute",
            "tolerationSeconds": 300
          }
        ],
        "priority": 0,
        "enableServiceLinks": true
      },
      "status": {
        "phase": "Running",
        "conditions": [
          {
            "type": "Initialized",
            "status": "True",
            "lastProbeTime": null,
            "lastTransitionTime": "2019-07-24T05:10:19Z"
          },
          {
            "type": "Ready",
            "status": "True",
            "lastProbeTime": null,
            "lastTransitionTime": "2019-07-24T05:10:26Z"
          },
          {
            "type": "ContainersReady",
            "status": "True",
            "lastProbeTime": null,
            "lastTransitionTime": "2019-07-24T05:10:26Z"
          },
          {
            "type": "PodScheduled",
            "status": "True",
            "lastProbeTime": null,
            "lastTransitionTime": "2019-07-24T05:10:20Z"
          }
        ],
        "hostIP": "172.17.0.33",
        "podIP": "10.44.0.1",
        "startTime": "2019-07-24T05:10:19Z",
        "containerStatuses": [
          {
            "name": "nginx",
            "state": {
              "running": {
                "startedAt": "2019-07-24T05:10:26Z"
              }
            },
            "lastState": {

            },
            "ready": true,
            "restartCount": 0,
            "image": "nginx:latest",
            "imageID": "docker-pullable://nginx@sha256:eb3320e2f9ca409b7c0aa71aea3cf7ce7d018f03a372564dbdb023646958770b",
            "containerID": "docker://d9a8a85f0173b51a6d2b952c03ec3621a1c9881cfadeed911bc4b1a078210734"
          }
        ],
        "qosClass": "BestEffort"
      }
    },
    {
      < data of next PODs omitted >

Therefore, it seems, that the designers of the API have chosen to suppress the „kind“ and „apiVersion“ from the POD information inside a PodList. Instead, we can deduct this information from the PodList.

Step 2: Explore the API

Step 2.1: Use strace to find the Cache Path

„strace“ is a diagnostic tool that helps us observing, which files the system is accessing when we apply a command. Let us apply that tool to a „kubectl“ command:

# optional for reviewing, what happens:
strace kubectl get nodes
# < output omitted >

# for a better overview, we filter the result:
strace kubectl get nodes 1>/dev/null 2> strace.out && grep openat strace.out

# output:
NAME     STATUS   ROLES    AGE     VERSION
master   Ready    master   5m36s   v1.14.0
node01   Ready    <none>   5m20s   v1.14.0
openat(AT_FDCWD, "/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/root/.kube/cache/discovery/172.17.0.54_6443/autoscaling/v1/serverresources.json", O_RDONLY|O_CLOEXEC) = 3

Here, we have redirected the STDERR to a file and then we grep the „openat“ traces from the file. You might need to apply the above command several times to see the file paths we are interested in: the cache discovery files. In our case, the cached files are located on

~/.kube/cache/discovery/172.17.0.54_6443/

The path will differ in your case.

Step 2.2: Explore the v1 API Resources

From there, we can explore the API in more detail. The API v1 commands are described in the file „v1/serverresources.json“ within the above path:

cat ~/.kube/cache/discovery/*/v1/serverresources.json

# output:
{"kind":"APIResourceList","apiVersion":"v1","groupVersion":"v1","resources":[{"name":"bindings","singularName":"","namespaced":true,"kind":"Binding","verbs":["create"]},{"name":"componentstatuses","singularName":"", ...

For better readability, we are beautifying the output by piping it through the „jq“ tool. Note that „jq“ is pre-installed on Katacoda already:

cat ~/.kube/cache/discovery/*/v1/serverresources.json | jq '.' 

# output:
{
  "kind": "APIResourceList",
  "apiVersion": "v1",
  "groupVersion": "v1",
  "resources": [
    {
      "name": "bindings",
      "singularName": "",
      "namespaced": true,
      "kind": "Binding",
      "verbs": [
        "create"
      ]
    },
    {
      "name": "componentstatuses",
      "singularName": "",
      "namespaced": false,
      "kind": "ComponentStatus",
      "verbs": [
        "get",
        "list"
      ],
      "shortNames": [
        "cs"
      ]
    },
    {
      "name": "configmaps",
      "singularName": "",
      "namespaced": true,
      "kind": "ConfigMap",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "cm"
      ]
    },
    {
      "name": "endpoints",
      "singularName": "",
      "namespaced": true,
      "kind": "Endpoints",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "ep"
      ]
    },
    {
      "name": "events",
      "singularName": "",
      "namespaced": true,
      "kind": "Event",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "ev"
      ]
    },
    {
      "name": "limitranges",
      "singularName": "",
      "namespaced": true,
      "kind": "LimitRange",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "limits"
      ]
    },
    {
      "name": "namespaces",
      "singularName": "",
      "namespaced": false,
      "kind": "Namespace",
      "verbs": [
        "create",
        "delete",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "ns"
      ]
    },
    {
      "name": "namespaces/finalize",
      "singularName": "",
      "namespaced": false,
      "kind": "Namespace",
      "verbs": [
        "update"
      ]
    },
    {
      "name": "namespaces/status",
      "singularName": "",
      "namespaced": false,
      "kind": "Namespace",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "nodes",
      "singularName": "",
      "namespaced": false,
      "kind": "Node",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "no"
      ]
    },
    {
      "name": "nodes/proxy",
      "singularName": "",
      "namespaced": false,
      "kind": "NodeProxyOptions",
      "verbs": [
        "create",
        "delete",
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "nodes/status",
      "singularName": "",
      "namespaced": false,
      "kind": "Node",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "persistentvolumeclaims",
      "singularName": "",
      "namespaced": true,
      "kind": "PersistentVolumeClaim",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "pvc"
      ]
    },
    {
      "name": "persistentvolumeclaims/status",
      "singularName": "",
      "namespaced": true,
      "kind": "PersistentVolumeClaim",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "persistentvolumes",
      "singularName": "",
      "namespaced": false,
      "kind": "PersistentVolume",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "pv"
      ]
    },
    {
      "name": "persistentvolumes/status",
      "singularName": "",
      "namespaced": false,
      "kind": "PersistentVolume",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "pods",
      "singularName": "",
      "namespaced": true,
      "kind": "Pod",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "po"
      ],
      "categories": [
        "all"
      ]
    },
    {
      "name": "pods/attach",
      "singularName": "",
      "namespaced": true,
      "kind": "PodAttachOptions",
      "verbs": [
        "create",
        "get"
      ]
    },
    {
      "name": "pods/binding",
      "singularName": "",
      "namespaced": true,
      "kind": "Binding",
      "verbs": [
        "create"
      ]
    },
    {
      "name": "pods/eviction",
      "singularName": "",
      "namespaced": true,
      "group": "policy",
      "version": "v1beta1",
      "kind": "Eviction",
      "verbs": [
        "create"
      ]
    },
    {
      "name": "pods/exec",
      "singularName": "",
      "namespaced": true,
      "kind": "PodExecOptions",
      "verbs": [
        "create",
        "get"
      ]
    },
    {
      "name": "pods/log",
      "singularName": "",
      "namespaced": true,
      "kind": "Pod",
      "verbs": [
        "get"
      ]
    },
    {
      "name": "pods/portforward",
      "singularName": "",
      "namespaced": true,
      "kind": "PodPortForwardOptions",
      "verbs": [
        "create",
        "get"
      ]
    },
    {
      "name": "pods/proxy",
      "singularName": "",
      "namespaced": true,
      "kind": "PodProxyOptions",
      "verbs": [
        "create",
        "delete",
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "pods/status",
      "singularName": "",
      "namespaced": true,
      "kind": "Pod",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "podtemplates",
      "singularName": "",
      "namespaced": true,
      "kind": "PodTemplate",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ]
    },
    {
      "name": "replicationcontrollers",
      "singularName": "",
      "namespaced": true,
      "kind": "ReplicationController",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "rc"
      ],
      "categories": [
        "all"
      ]
    },
    {
      "name": "replicationcontrollers/scale",
      "singularName": "",
      "namespaced": true,
      "group": "autoscaling",
      "version": "v1",
      "kind": "Scale",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "replicationcontrollers/status",
      "singularName": "",
      "namespaced": true,
      "kind": "ReplicationController",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "resourcequotas",
      "singularName": "",
      "namespaced": true,
      "kind": "ResourceQuota",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "quota"
      ]
    },
    {
      "name": "resourcequotas/status",
      "singularName": "",
      "namespaced": true,
      "kind": "ResourceQuota",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "secrets",
      "singularName": "",
      "namespaced": true,
      "kind": "Secret",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ]
    },
    {
      "name": "serviceaccounts",
      "singularName": "",
      "namespaced": true,
      "kind": "ServiceAccount",
      "verbs": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "sa"
      ]
    },
    {
      "name": "services",
      "singularName": "",
      "namespaced": true,
      "kind": "Service",
      "verbs": [
        "create",
        "delete",
        "get",
        "list",
        "patch",
        "update",
        "watch"
      ],
      "shortNames": [
        "svc"
      ],
      "categories": [
        "all"
      ]
    },
    {
      "name": "services/proxy",
      "singularName": "",
      "namespaced": true,
      "kind": "ServiceProxyOptions",
      "verbs": [
        "create",
        "delete",
        "get",
        "patch",
        "update"
      ]
    },
    {
      "name": "services/status",
      "singularName": "",
      "namespaced": true,
      "kind": "Service",
      "verbs": [
        "get",
        "patch",
        "update"
      ]
    }
  ]
}

As a reference, we have printed the full, long output with a scrollbar. In your case, you will find nicer coloring as follows:

Kubernetes API Server Resources

Step 2.3: Explore short Names

Many resources have short names. Let us get a list of them all:

cat ~/.kube/cache/discovery/*/v1/serverresources.json | jq '.' | awk '/kind/ {print} /short/ {getline; print}'

# output:
  "kind": "APIResourceList",
      "kind": "Binding",
      "kind": "ComponentStatus",
        "cs"
      "kind": "ConfigMap",
        "cm"
      "kind": "Endpoints",
        "ep"
      "kind": "Event",
        "ev"
      "kind": "LimitRange",
        "limits"
      "kind": "Namespace",
        "ns"
      "kind": "Namespace",
      "kind": "Namespace",
      "kind": "Node",
        "no"
      "kind": "NodeProxyOptions",
      "kind": "Node",
      "kind": "PersistentVolumeClaim",
        "pvc"
      "kind": "PersistentVolumeClaim",
      "kind": "PersistentVolume",
        "pv"
      "kind": "PersistentVolume",
      "kind": "Pod",
        "po"
      "kind": "PodAttachOptions",
      "kind": "Binding",
      "kind": "Eviction",
      "kind": "PodExecOptions",
      "kind": "Pod",
      "kind": "PodPortForwardOptions",
      "kind": "PodProxyOptions",
      "kind": "Pod",
      "kind": "PodTemplate",
      "kind": "ReplicationController",
        "rc"
      "kind": "Scale",
      "kind": "ReplicationController",
      "kind": "ResourceQuota",
        "quota"
      "kind": "ResourceQuota",
      "kind": "Secret",
      "kind": "ServiceAccount",
        "sa"
      "kind": "Service",
        "svc"
      "kind": "ServiceProxyOptions",
      "kind": "Service",

If you scroll down a little, you will find the short name for Node: „no“. Therefore, we now know that the get nodes command can be abbreviated as follows:

kubectl get no

# output:
NAME     STATUS   ROLES    AGE   VERSION
master   Ready    master   43m   v1.14.0
node01   Ready    <none>   43m   v1.14.0

This is especially helpful for longer names as follows:

# list of often used short names:
ep  = EndPoint
ns  = NameSpace
pvc = PersistentVolumeClaim
pv  = PersistentVolume
rc  = ReplicationController
svc = Service

Note: kubectl command are case sensitive on short names, but they are not case sensitive on long names. Moreover, long names do not require, but allow plural ’s‘ to be appended:

# OK
kubectl get Nodes
kubectl get Node
kubectl get noDE
kubectl get no

# Not OK:
kubectl get Nod
kubectl get No

Step 2.4: Other Versions

We also can explore other API versions than the „v1“:

apps and extension API versions

For exploring the names and short names, we can apply our awk command again:

cat ~/.kube/cache/discovery/*/apps/v1/serverresources.json \
  | jq '.' \
  | awk '/kind/ {print} /short/ {getline; print}'

# output:
  "kind": "APIResourceList",
      "kind": "ControllerRevision",
      "kind": "DaemonSet",
        "ds"
      "kind": "DaemonSet",
      "kind": "Deployment",
        "deploy"
      "kind": "Scale",
      "kind": "Deployment",
      "kind": "ReplicaSet",
        "rs"
      "kind": "Scale",
      "kind": "ReplicaSet",
      "kind": "StatefulSet",
        "sts"
      "kind": "Scale",
      "kind": "StatefulSet",

In this example, we find that our beloved DaemonSet also has a short name: „ds“

Step 3: API Access via Token

In this chapter, we will test Kubernetes API access via bearer token. We have decided to extend the API article accordingly, even though this topic has been moved out to the next chapter within LSF458.

Step 3.1: Retrieve the Namespace Token

First. let us retrieve the token from namespace „default“:

NS=default
TOKEN=$(kubectl get secrets -n $NS -o json | jq -r '.items[0].data.token' | base64 -d)
echo $TOKEN

# output:
eyJhb ( output omitted here ...) 7B8R3xw

The output will look different in your case.

Step 3.2: Get API URL

Let us now retrieve the API URL from the configuration file

API_URL=$(kubectl config view | grep server | awk '{print $2}')
echo $API_URL

# output:
https://172.17.0.34:6443

Step 3.3: Access the API

All we need to access the API is the API URL and the token. Let us access the API now. Let us ask for the list of PODs, as we have done that above:

curl -k -H "Authorization: Bearer $TOKEN" $API_URL/api/v1/pods

Note: if we omit the -k or –insecure option, curl will complain that the certificate is not secure. We have not needed this in Step 1 since we have provided the cacert with the command. If you want to avoid the -k option, you can also choose to do so here as well:

item=certificate-authority-data;  
grep $item ~/.kube/config | awk '{print $2}' | base64 -d > $item.pem

curl --cacert ./certificate-authority-data.pem \
     -H "Authorization: Bearer $TOKEN" \
     $API_URL/api/v1/pods

If everything goes right, we will get the following output in both cases:

{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/pods",
    "resourceVersion": "4864"
  },
  "items": [
    {
      "metadata": {
        "name": "coredns-fb8b8dccf-sctlh",
        "generateName": "coredns-fb8b8dccf-",
        "namespace": "kube-system",
        "selfLink": "/api/v1/namespaces/kube-system/pods/coredns-fb8b8dccf-sctlh",
        "uid": "d655d680-ae46-11e9-9c0f-0242ac110014",
        "resourceVersion": "482",
        "creationTimestamp": "2019-07-24T19:11:27Z",
        "labels": {
          "k8s-app": "kube-dns",
          "pod-template-hash": "fb8b8dccf"
        },

< output omitted >

Step 4: Explore APIs

As we already have seen above, there are more APIs available beyond the /api/v1 API. We now can use the token access to retrieve a full list of APIs:

curl -k -s -H "Authorization: Bearer $TOKEN" \
     $API_URL/apis | grep "^          \"groupVersion"

# output:
          "groupVersion": "apiregistration.k8s.io/v1",
          "groupVersion": "apiregistration.k8s.io/v1beta1",
          "groupVersion": "extensions/v1beta1",
          "groupVersion": "apps/v1",
          "groupVersion": "apps/v1beta2",
          "groupVersion": "apps/v1beta1",
          "groupVersion": "events.k8s.io/v1beta1",
          "groupVersion": "authentication.k8s.io/v1",
          "groupVersion": "authentication.k8s.io/v1beta1",
          "groupVersion": "authorization.k8s.io/v1",
          "groupVersion": "authorization.k8s.io/v1beta1",
          "groupVersion": "autoscaling/v1",
          "groupVersion": "autoscaling/v2beta1",
          "groupVersion": "autoscaling/v2beta2",
          "groupVersion": "batch/v1",
          "groupVersion": "batch/v1beta1",
          "groupVersion": "certificates.k8s.io/v1beta1",
          "groupVersion": "networking.k8s.io/v1",
          "groupVersion": "networking.k8s.io/v1beta1",
          "groupVersion": "policy/v1beta1",
          "groupVersion": "rbac.authorization.k8s.io/v1",
          "groupVersion": "rbac.authorization.k8s.io/v1beta1",
          "groupVersion": "storage.k8s.io/v1",
          "groupVersion": "storage.k8s.io/v1beta1",
          "groupVersion": "admissionregistration.k8s.io/v1beta1",
          "groupVersion": "apiextensions.k8s.io/v1beta1",
          "groupVersion": "scheduling.k8s.io/v1",
          "groupVersion": "scheduling.k8s.io/v1beta1",
          "groupVersion": "coordination.k8s.io/v1",
          "groupVersion": "coordination.k8s.io/v1beta1",
          "groupVersion": "node.k8s.io/v1beta1",

Note that there is a batch API (in blue), which we will explore in more detail in our next blog post. There, we will test the Kubernetes Jobs and CronJob functions.

Step 5: Access the API from inside a Container

In each container, a secret volume with a service account is attached.

(master) # kubectl config view | grep server | awk '{print $2}'
# output: https://172.17.0.21:6443

(master) # kubectl run -it --image=centos bash
(containter) # cd /var/run/secrets/kubernetes.io/serviceaccount/
(containter) # curl --cacert ./ca.crt -H "Authorization: Bearer $(cat ./token)" https://172.17.0.21:6443/api/v1/pods

# output:
{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/pods",
    "resourceVersion": "5221"
  },
  "items": [
    {
      "metadata": {
        "name": "bash-5d65698d48-xtl2r",
        "generateName": "bash-5d65698d48-",
        "namespace": "default",
        "selfLink": "/api/v1/namespaces/default/pods/bash-5d65698d48-xtl2r",
        "uid": "6f9cd8d7-ae57-11e9-b206-0242ac110015",
        "resourceVersion": "4378",
        "creationTimestamp": "2019-07-24T21:10:16Z",
        "labels": {
          "pod-template-hash": "5d65698d48",
          "run": "bash"
< output omitted >

Note: the API URL most probably will differ in your case.

We have accessed the API successfully also from within the container.

TODO: Clarify the following question; Does that mean that each and every container has full access to the Kubernetes API?

Step 6: Access the API via API Proxy

Another method to access the API is via API proxy. For that, we start a proxy on the master (the node does not have the needed configuration in order to successfully access the cluster’s API).

kubectl proxy
Starting to serve on 127.0.0.1:8001

The proxy is running in the foreground, so the terminal seems to be frozen now. We open a second terminal on the same machine and access the API without any token or key:

curl 127.0.0.1:8001/api/v1/pods

# output:
{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/pods",
    "resourceVersion": "3659"
  },
  "items": [
    {
      "metadata": {
        "name": "coredns-fb8b8dccf-p2d7t",
        "generateName": "coredns-fb8b8dccf-",
        "namespace": "kube-system",
        "selfLink": "/api/v1/namespaces/kube-system/pods/coredns-fb8b8dccf-p2d7t",
        "uid": "b211cb91-ae50-11e9-b206-0242ac110015",
        "resourceVersion": "483",
        "creationTimestamp": "2019-07-24T20:22:01Z",
        "labels": {
          "k8s-app": "kube-dns",
          "pod-template-hash": "fb8b8dccf"
< output omitted >

If you want to allow access from any other machine, you need to add the address and … option

kubectl proxy --address '0.0.0.0' --disable-filter=true

Since anybody can access your API now, a warning will appear. Now, you can access the API from node01:

curl -s 172.17.0.21:8001/api/v1/pods | head -n 20

# output:
{
  "kind": "PodList",
  "apiVersion": "v1",
  "metadata": {
    "selfLink": "/api/v1/pods",
    "resourceVersion": "4065"
  },
  "items": [
    {
      "metadata": {
        "name": "coredns-fb8b8dccf-p2d7t",
        "generateName": "coredns-fb8b8dccf-",
        "namespace": "kube-system",
        "selfLink": "/api/v1/namespaces/kube-system/pods/coredns-fb8b8dccf-p2d7t",
        "uid": "b211cb91-ae50-11e9-b206-0242ac110015",
        "resourceVersion": "483",
        "creationTimestamp": "2019-07-24T20:22:01Z",
        "labels": {
          "k8s-app": "kube-dns",
          "pod-template-hash": "fb8b8dccf"

In any of the above configurations, we have succeeded to access the Kubernetes API.

Previous:

Node Maintenance

Next:

Kubernetes Jobs and CronJobs (Link to be provided)

5 comments

Comments

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.