Containers / Kubernetes / Tools

Configuring the Ultimate Development Environment for Kubernetes

26 Aug 2016 8:21am, by

There is a general perception that Kubernetes is meant only for professionals with advanced infrastructure management skills. While there is a bit of truth in it, there are multiple ways to configure Kubernetes without getting overwhelmed.

With each major release, Kubernetes is getting closer to the developer community. In its current form, it is extremely simple to turn your existing Mac or Windows machine into a powerful Kubernetes development environment. I am playing with Kubernetes on the new 12” Gold MacBook with 8GB RAM and 256GB storage powered by a 1.1GHz Intel Core M3 processor. By no means this is meant to be a developer workstation but, with Minikube, I am able to carry a full-blown Kubernetes environment with me.

This tutorial aims at enabling Docker developers to easily get started with Kubernetes. It will walk you through all the prerequisites, and the basic concepts to deploy your first microservices application powered by Kubernetes. Though this is based on Mac, it will work with Microsoft Windows 10 with just a few modifications.

What Do We Need?

First things first, let’s make an inventory of the tools that we need.

  1. VirtualBox: This is the core foundation on which we will configure the environment. Download the latest version of VirtualBox.
  2. Docker Toolbox: This is the essential component of our toolchain. We will extensively use Docker Machine for the configuration. Download the stable version of Docker Toolbox.
  3. DVM: It can get frustrating to deal with disparate Docker versions running within the same environment. Like Node Version Manager and Ruby Version Manager, this nifty tool makes it super simple to switch between multiple versions of Docker CLI. We will explore this tool in the later part of the tutorial. Follow these instructions on GitHub to get DVM on your machine.
  4. Minikube: All good things in life come in small packages, and Kubernetes is not an exception. Minikube is a tiny VM that comes bundled with a single node Kubernetes cluster. The best thing is that it can run on any machine that is capable of launching VirtualBox VMs. Get the latest binary from GitHub.
  5. Kubectl: Kubectl is the command line tool to interact with the Kubernetes cluster. Follow the instructions to download and install it.
  6. Docker for Windows or Mac [Optional]: Since we are dealing with Docker Machine for creating Swarm cluster, we don’t really need native installers for Windows and Mac. But if you prefer, you can also have it run side-by-side. Make sure that you are grabbing the latest installer.

What Do We Get?

Most of the developers have an isolated development and testing environments for Docker and Kubernetes. This makes it frustrating to quickly build and deploy from source code. The Dockerfile targeting the core engine is always different from the Kubernetes environment. This forces us to push the images to the Docker Hub or a public registry and pull them in Kubernetes nodes, even when they are running within the same environment. While this provides a good isolation for production setup, it can get pretty annoying during development and testing.

In our environment, we will create a central registry that Docker and Kubernetes will share. Every image that we build will become instantly available to the Kubernetes cluster. This brings parity between Docker and Kubernetes environments.

This setup makes it easy to deploy applications in parallel running in Swarm and Kubernetes. Each time an image gets pushed to the registry, it immediately becomes available to both the Swarm cluster and Kubernetes.

This tutorial has some tips and tricks that will help you get a grip on managing disparate Docker environments.

The illustration below explains the setup and the configuration.



Let’s start by creating a central registry that we will access during development and testing. We will then configure Docker and Kubernetes to use the registry.

Step 1: Docker Private Registry

Create a new Docker Machine that acts as the registry:

docker-machine create -d virtualbox registry

Create the directory to hold images:

docker-machine ssh registry "mkdir ~/data"

Point the Docker CLI to the machine:

eval $(docker-machine env registry)

Pull and run the registry image:

docker run -d -p 80:5000 --restart=always --name registry   -v /home/docker/data:/var/lib/registry registry:2

Finally, make note of the IP address of Docker Machine running the registry:

REG_IP=`docker-machine ip registry`

Step 2: Docker Dev Machine

Create a new Docker Machine that will be our default development environment:

docker-machine create -d virtualbox dev

Configure the Docker Engine to point to the private insecure registry:

docker-machine ssh dev

Open /var/lib/boot2docker/profile in the editor and add the following line.  Make sure that you included the IP address of the Docker Machine hosting the registry:

EXTRA_ARGS="--insecure-registry <REG_IP>:80"

Optionally, you can run the following single line command to do the same:

docker-machine ssh dev 'sudo sh -c "echo \"EXTRA_ARGS=\\\"--insecure-registry '$REG_IP':80\\\"\" >> /var/lib/boot2docker/profile"'

Exit from the VM and restart the machine:

docker-machine restart dev

Point the Docker CLI to the dev machine:

eval $(docker-machine env dev)

Pull an image, tag it, and push it the private registry:

docker pull hello-world

docker tag hello-world $REG_IP:80/hello-world

docker push $REG_IP:80/hello-world

This step successfully configured the Docker engine to use the insecure private registry that we created in the previous step.

Step 3: Kubernetes Single Node Cluster

With the Docker private registry and development machine in place, let’s go ahead and configure Kubernetes. This step assumes that you followed the steps mentioned in the Minikube installation guide.

Start the Minikube VM:

minikube start --vm-driver="virtualbox" --insecure-registry="$REG_IP":80

A quick note on the configuration of private registry: Minikube can be pointed to the insecure registry during the creation of VM. It is exactly the same as the steps performed in configuring Docker Machine to use the private registry. In fact, Docker Machine also supports this configuration through the — engine-opt switch. Refer to the documentation for more details.

Check the status of Minikube:

minikube status

Check if kubectl is correctly configured:

kubectl get cs

Step 4: Test the Workflow

With the testbed in place, let’s now try the entire round trip: Build an image on Docker development machine and deploy it in Kubernetes.

We will push the standard Apache image from the Docker Hub, modify it on the dev machine, commit and push the new image to the local registry.

The same image will be then deployed on Kubernetes.

Build on Docker

Set the CLI to Docker development machine:

eval $(docker-machine env dev)

Pull, tag, and push the image:

docker pull httpd
docker run -d --name myweb --hostname myweb -p 80:80  httpd

docker exec myweb bash -c "echo Kubernetes Rocks' > '/usr/local/apache2/htdocs/index.html'"

docker commit myweb $REG_IP:80/myweb

docker push $REG_IP:80/myweb

Deploy and Run in Kubernetes

Point the Docker CLI to minikube:

eval $(minikube docker-env)

When you try to access the Docker engine on minikube, you will get an error about version mismatch between the client and server:

Error response from daemon: client is newer than server (client API version: 1.24, server API version: 1.23)

This is where we will take the help of Docker Version Manager. First, let’s check the Docker version on Minikube:

minikube ssh docker version

We can capture the server version into a variable, which will be passed to DVM:

DOCKER_VER=`minikube ssh "docker version --format '{{.Server.Version}}'"`

Run the following command to switch to the correct version of Docker CLI:

dvm use $DOCKER_VER

Now you can use the same client to target Docker Dev machine and Kubernetes machine.

Let’s pull the image that we uploaded to the private registry:

docker pull $REG_IP:80/myweb

Finally, let’s create a Kubernetes deployment and expose the container:

kubectl run my-web --image=$REG_IP:80/myweb --port=80

kubectl expose deployment my-web --target-port=80 --type=NodePort

Grab the NodePort and access the service endpoint:

PORT=$(kubectl get svc my-web  -o go-template='{{(index .spec.ports 0).nodePort}}')

IP=$(minikube ip)

curl $IP:$PORT

The above command should print “Kubernetes Rocks.”

Let’s clean up by delete the deployment and service:

kubectl delete service my-web

kubectl delete deployment my-web

You can download the scripts for all the steps from GitHub.

This setup has a few caveats:

  • It doesn’t use secure registry access but it can be easily configured
  • Images are not persisted on Mac or Windows host. If the Docker machine hosting the registry crashes, you will lose images. This can be avoided by mounting local volumes from the host machine and pointing the registry volume to it.
  • You can use Docker for Mac and configure the insecure registry option to pull images from the local registry.

If you want to get started with Kubernetes, please register for the free webinar that I am co-hosting, on Monday, August 29. We will show you how to setup your development environment, and deploy your first application.


A digest of the week’s most important stories & analyses.

View / Add Comments

Please stay on topic and be respectful of others. Review our Terms of Use.