This blog post shows a quick way how to install Kubernetes, a Docker orchestration framework published by Google, as a set of Docker containers. With that you can circumvent the hassle you may run into while trying to install Kubernetes natively.

Note: previous blogs in the DDDocker series can be found on LinkedIn; more recent blog posts on Docker are found here on WordPress.

Versions

v0.1 (draft) 2015-08-12: described 3 ways of installing Kubernetes, but they either failed or they led to problems later on, because ‚make‘ and other commands were missing on the machine running the kubectl client (Windows related problem). Now this installation version is moved to the appendix, since it might still be needed later on, if testing other procedures on Linux.
v0.2 (draft) 2015-08-14: added coarse outline of my (now successful) 4th attempt
v1.0 2015-08-17: full description of the successful attempt; I have moved the old, problematic attempt into the appendix.
v1.1 2015-08-17: added a subchapter „Networking Challenges“, which shows how to route from the Windows host to the service.
v1.2 2015-08-18: moved the page to wordpress.com, since LinkedIn blogs was not available to all of my colleagues.
v1.3 2016-07-11: moved the documentation of the unsuccessful attempts to the end of the document

Introduction

What is Kubernetes all about? You might love this intro: The Illustrated Children’s Guide to Kubernetes.

In the last blog I have investigated some low level container orchestration using fleet, which calls itself a „simple distributed init system“, but we could show that it offers possibilities to

  • define container-based services and
  • monitor the health of Docker hosts
  • automatic restart of containers on other hosts, if a Docker host fails.

For those looking for more complex scheduling requirements or a first-class container orchestration system, Kubernetes of google is recommended. Let us explore, what Kubernetes adds to fleet’s capabilities, how to install it and how to test its core features. Kubernetes is a core element of other, more complete Container frameworks, e.g. Red Hat’s OpenShift Container Platform (a.k.a. OpenShift Enterprise 3.x).

Kubernetes Architecture

In the architecture consists of a master docker node and one or more minion nodes. In our example the master node offers:

  1. kubectl, i.e. the kube client
  2. the REST API with authentication, Replication Controller and Scheduler
  3. the kubelet info service, i.e. the service, which talks to the other docker hosts

Depending on the size of the solution, the functions can be spread over different docker hosts.

2015.08.05-18_55_22-hc_001

The minion docker hosts that are hosting pods offer following functions:

  1. kublet, i.e. the kube agent the kubelet info service talks to
  2. cAdvisor, which is used to monitor containers
  3. a proxy, which offers an abstraction layer for the communication with pods, see the description of pods.

Pods:

  • are a set of containers on a single docker host
  • each pod is assigned an IP address
  • communication between pods is performed via a proxy, which is the abstraction layer offering the pod’s IP address from outside

kubectl is the client talking to a REST API, which in turn talks to the kublet info service, which in turn talks to the pods via local kublet agents.

etcd is used as a distributed key storage system. I guess, host clustering is done via the etcd discovery service (t.b.v.).

Installation of Kubernetes

…on Ubuntu Docker host (works well)…

We are following the instructions: Running Kubernetes locally via Docker on a Ubuntu VM created via Vagrant.

Installation of Ubuntu using Vagrant

Prerequisites:

  • Vagrant, Virtualbox are installed

If you are operating behind a HTTP proxy, set the http_proxy and https_proxy variables accordingly (please replace the name/IP address and port that it matches your environment):

export http_proxy="http://proxy.example.com:8080"
export https_proxy="http://proxy.example.com:8080"

Create and initialize a Vagrant working directory:

mkdir ubuntu-trusty64-docker; cd ubuntu-trusty64-docker
vagrant init williamyeh/ubuntu-trusty64-docker

Start and connect to the VM:

vagrant up
vagrant ssh

Start Kubernetes Docker Containers

If you are operating behind a proxy, set the http_proxy and https_proxy variables accordingly and add those variables also to the docker environment:

# perform this section, if you are behind a HTTP proxy, but replace IP address and port to match your environment:
export http_proxy="http://proxy.example.com:8080"
export https_proxy="http://proxy.example.com:8080"
sudo vi  /etc/default/docker
# add the export commands above to the file /etc/default/docker (with sudo vi  /etc/default/docker) and restart the docker service:
sudo service docker restart

Install kubectl:

sudo wget https://storage.googleapis.com/kubernetes-release/release/v1.0.3/bin/linux/amd64/kubectl -O /tmp/kubectl
sudo cp /tmp/kubectl /usr/local/bin/; sudo chmod +x /usr/local/bin/kubectl

Stop the cAdvisor, since it would lead to a clash in the port assignment:

docker ps | grep -i cadvisor | grep ':8080->' | awk '{print $1}' | xargs --no-run-if-empty docker stop

With docker ps, make sure that no docker container is running at this point:

Now you can follow the instructions on https://github.com/pires/kubernetes-vagrant-coreos-cluster/blob/master/README.md, like follows:

Step One: Run etcd

docker run --net=host -d gcr.io/google_containers/etcd:2.0.9 /usr/local/bin/etcd --addr=127.0.0.1:4001 --bind-addr=0.0.0.0:4001 --data-dir=/var/etcd/data

Step Two: Run the master

docker run --net=host -d -v /var/run/docker.sock:/var/run/docker.sock gcr.io/google_containers/hyperkube:v0.21.2 /hyperkube kubelet --api_servers=http://localhost:8080 --v=2 --address=0.0.0.0 --enable_server --hostname_override=127.0.0.1 --config=/etc/kubernetes/manifests

Step Three: Run the service proxy

docker run -d --net=host --privileged gcr.io/google_containers/hyperkube:v0.21.2 /hyperkube proxy --master=http://127.0.0.1:8080 --v=2

Test it out

kubectl get nodes

Run an application

kubectl -s http://localhost:8080 run-container nginx --image=nginx --port=80

Now run

docker ps

you should see nginx running. You might need to wait a few minutes for the image to get pulled:

Expose it as a service

kubectl expose rc nginx --port=80

If CLUSTER_IP is blank (Known kubernetes issue #10836) run the following command to obtain it:

kubectl get svc nginx

Test the web server:

export no_proxy="<insert-ip-from-above-here>"
curl <insert-ip-from-above-here>

Bingo! The NginX web server is up and running!

Accessing the Service from remote

The service is reachable from the Vagrant Linux host. However, the service cannot be reached from my Windows machine yet.

The problem can be described like follows:

  1. Kubernetes is automatically fetching an IP address from a pool (defined where?) for each service. In case of the Nginx service, this was the IP address 10.0.0.146.
  2. The address is not owned by the Vagrant Linux VM, as can be seen with an ifconfig.
  3. In Vagrant, per default, we have no public IP address. However, Vagrant offers the possibility to map a VM’s IP address and port to a port of the host (= Windows host in my case). However, I have not found any possibility to map a host’s IP:port pair to an IP and port that is not owned by the VM.

We have two possibilities to resolve the problem:

1) Chained port mapping (only theory; not tested yet):

  • In Vagrant map the host’s port to an IP:port pair owned by the VM
  • In the VM, using e.g. iptables NAT function to map the IP:port pair to the service’s IP:port pair

2) Create an additional, reachable interface for the VM and route the service IP address to this public interface

  • In the Vagrantfile, add e.g. the line
    config.vm.network „private_network“, ip: „192.168.33.10“ -> this will automatically create the interface eth1 in a new host-only network. You need to issue „vagrant reload –provision“ to activate this setting.
  • On the (Windows) host, add a route to the network, which matches the pool Kubernetes is choosing the service IP addresses from. In my case, I have added the route using the command
    route add 10.0.0.0 mask 255.255.255.0 192.168.33.10

With this, the Nginx service becomes reachable (now on another, randomly chosen IP address 10.0.0.19, since I have restarted the host):

Perfect! The service is now available also from the Windows host.
In a real world example, external load balancers will map the externally visible IP:port pair to the service’s IP:Port pair; and the IP address and port will be chosen statically. Routing needs to take care that the service is reached, no matter, on which host it is located. This is something, I need to explore in more detail an another post: how can we connect several Minions to the same network, and can we make sure that the IP packets are routed to the right Minion? Gratuitous ARP?

Appendix: Installation of Kubernetes on CoreOS (removed)

This was a log of my efforts, but it lead to a dead end (installation of kubectl on CoreOS is not supported and running kubectl in a docker container did not lead to the desired results), so I have removed it; still available on request as revision 18 …

Appendix: Attempts to install Kubernetes (including unsuccessful attempts)

Installation of Kubernetes on Windows seems to be hard, as can be seen with the first three unsuccessful installation attempts. However, I have found a fairly automated way of installing Kubernetes as a set of Docker containers by using Vagrant and a base image that has docker already installed on an Ubuntu VM. This is described as successful attemt 4) below and is described in more detail in the main part of this blog.

UNSUCCESSFUL ATTEMPTS:

1) Multi-node CoreOS cluster installation on the Getting Started CoreOS page
I had to try 3 times, until the kubectl client was downloaded and installed correctly. And when trying to start my first NginX example, I found myself in a dead end: the example(s) require normal Linux commands line „make“, but CoreOS neither support those commands, nor allows to install them.

2) Running Kubernetes locally via Docker
this is supposed to be the quick way for an evaluation installation, since we only need to download and run pre-installed Docker images. Not so this time: here, I run into the problem that kubectl client cannot be installed on my boot2docker host. When I try to use one of the kubectl docker images, I always get an error that 127.0.0.1:8080 cannot be reached.

3) Running Kubernetes locally via Docker within an Ubuntu docker container on Windows boot2docker does not work either: if I install kubectl on that container, kubectl cluster-info always returns a success message no matter, which server IP address I specify and no matter, whether the kubernetes containers are up and running or not. Shoot.

SUCCESSFUL ATTEMPT:

4) SUCCESS: Running Kubernetes locally via Docker within an Ubuntu VM has succeeded finally: I had created an Ubuntu VM using Vagrant with the image ubuntu-trusty64-docker from the Vagrant boxes repository and I have downloaded kubectl v1.0.1 into /usr/local/bin of that image. First, I had the problem that kubectl always had returned an error that I could not find on Google, saying that it had received the string „Supported versions: [v1.0,…]“ or similar. Then I had found out that kubectl is connecting to http://localhost:8080, which was occupied already with a docker image google/cadvisor, that was up and running in the Ubuntu Vagrant image I had used. After finding this docker image with „docker ps“ and stopping it with „docker stop <container-id>“, kubectl worked as expected. Now „wget -qO- http://localhost:8080/ | less“ returns a list of paths in json format and all kubectl commands on the instruction page are working as expected. That was hard work. My assumption that it would work on Ubuntu was correct. I will troubleshoot in more details, why it had not worked in one of the other ways.

 

35 comments

  1. Sight Care is a daily supplement proven in clinical trials and conclusive science to improve vision by nourishing the body from within. The Sight Care formula claims to reverse issues in eyesight, and every ingredient is completely natural.

  2. Cortexi is a completely natural product that promotes healthy hearing, improves memory, and sharpens mental clarity. Cortexi hearing support formula is a combination of high-quality natural components that work together to offer you with a variety of health advantages, particularly for persons in their middle and late years. Cortex not only improves hearing but also decreases inflammation, eliminates brain fog, and gives natural memory protection.

  3. Boostaro increases blood flow to the reproductive organs, leading to stronger and more vibrant erections. It provides a powerful boost that can make you feel like you’ve unlocked the secret to firm erections

  4. Puravive introduced an innovative approach to weight loss and management that set it apart from other supplements. It enhances the production and storage of brown fat in the body, a stark contrast to the unhealthy white fat that contributes to obesity.

  5. FitSpresso stands out as a remarkable dietary supplement designed to facilitate effective weight loss. Its unique blend incorporates a selection of natural elements including green tea extract, milk thistle, and other components with presumed weight loss benefits.

  6. Prostadine is a dietary supplement meticulously formulated to support prostate health, enhance bladder function, and promote overall urinary system well-being. Crafted from a blend of entirely natural ingredients, Prostadine draws upon a recent groundbreaking discovery by Harvard scientists. This discovery identified toxic minerals present in hard water as a key contributor to prostate issues.

  7. Neotonics is an essential probiotic supplement that works to support the microbiome in the gut and also works as an anti-aging formula. The formula targets the cause of the aging of the skin.

  8. Cortexi is an effective hearing health support formula that has gained positive user feedback for its ability to improve hearing ability and memory. This supplement contains natural ingredients and has undergone evaluation to ensure its efficacy and safety. Manufactured in an FDA-registered and GMP-certified facility, Cortexi promotes healthy hearing, enhances mental acuity, and sharpens memory.

  9. Dentitox Pro is a liquid dietary solution created as a serum to support healthy gums and teeth. Dentitox Pro formula is made in the best natural way with unique, powerful botanical ingredients that can support healthy teeth.

  10. Gorilla Flow is a non-toxic supplement that was developed by experts to boost prostate health for men. It’s a blend of all-natural nutrients, including Pumpkin Seed Extract Stinging Nettle Extract, Gorilla Cherry and Saw Palmetto, Boron, and Lycopene.

  11. Nervogen Pro, A Cutting-Edge Supplement Dedicated To Enhancing Nerve Health And Providing Natural Relief From Discomfort. Our Mission Is To Empower You To Lead A Life Free From The Limitations Of Nerve-Related Challenges. With A Focus On Premium Ingredients And Scientific Expertise.

  12. Endo Pump Male Enhancement is a dietary supplement designed to assist men improve their sexual performance. This natural formula contains a potent blend of herbs and nutrients that work together to improve blood flow

  13. TerraCalm is an antifungal mineral clay that may support the health of your toenails. It is for those who struggle with brittle, weak, and discoloured nails. It has a unique blend of natural ingredients that may work to nourish and strengthen your toenails.

  14. SonoVive™ is an all-natural supplement made to address the root cause of tinnitus and other inflammatory effects on the brain and promises to reduce tinnitus, improve hearing, and provide peace of mind.

  15. Introducing FlowForce Max, a solution designed with a single purpose: to provide men with an affordable and safe way to address BPH and other prostate concerns. Unlike many costly supplements or those with risky stimulants, we’ve crafted FlowForce Max with your well-being in mind. Don’t compromise your health or budget – choose FlowForce Max for effective prostate support today!

  16. Claritox Pro™ is a natural dietary supplement that is formulated to support brain health and promote a healthy balance system to prevent dizziness, risk injuries, and disability. This formulation is made using naturally sourced and effective ingredients that are mixed in the right way and in the right amounts to deliver effective results.

  17. Glucofort Blood Sugar Support is an all-natural dietary formula that works to support healthy blood sugar levels. It also supports glucose metabolism. According to the manufacturer, this supplement can help users keep their blood sugar levels healthy and within a normal range with herbs, vitamins, plant extracts, and other natural ingredients.

  18. The Quietum Plus supplement promotes healthy ears, enables clearer hearing, and combats tinnitus by utilizing only the purest natural ingredients. Supplements are widely used for various reasons, including boosting energy, lowering blood pressure, and boosting metabolism.

  19. TropiSlim is a unique dietary supplement designed to address specific health concerns, primarily focusing on weight management and related issues in women, particularly those over the age of 40. TropiSlim targets a unique concept it refers to as the „menopause parasite“ or K-40 compound, which is purported to be the root cause of several health problems, including unexplained weight gain, slow metabolism, and hormonal imbalances in this demographic.

  20. GlucoBerry is one of the biggest all-natural dietary and biggest scientific breakthrough formulas ever in the health industry today. This is all because of its amazing high-quality cutting-edge formula that helps treat high blood sugar levels very naturally and effectively.

  21. Kerassentials are natural skin care products with ingredients such as vitamins and plants that help support good health and prevent the appearance of aging skin. They’re also 100% natural and safe to use. The manufacturer states that the product has no negative side effects and is safe to take on a daily basis. Kerassentials is a convenient, easy-to-use formula.

Comments

Diese Website verwendet Akismet, um Spam zu reduzieren. Erfahre mehr darüber, wie deine Kommentardaten verarbeitet werden.