What news from AWS re:Invent last week will have the most impact on you?
Amazon Q, an AI chatbot for explaining how AWS works.
Super-fast S3 Express storage.
New Graviton 4 processor instances.
Emily Freeman leaving AWS.
I don't use AWS, so none of this will affect me.

Configuring Kubernetes Cluster Federation to Create a Global Deployment

Jun 8th, 2018 9:00am by
Featued image for: Configuring Kubernetes Cluster Federation to Create a Global Deployment
Feature image via Pixabay.

One of the advantages of running workloads in Kubernetes is the ease of configuring desired state. Once a Replica Set, a StatefulSet, or a Deployment is configured to run a certain number of Pods, Kubernetes control plane will ensure that those many instances are available. Managed Kubernetes offerings such as Google Kubernetes Engine and Azure Kubernetes Service offer Nodes in high availability mode, which delivers increased resiliency.

Cluster federation in Kubernetes takes the concept of high availability to the next level by making clusters resilient. Multiple distributed clusters can be federated to ensure that the workload is available in at least one cluster. The best way of understanding cluster federation is to visualize a meta-cluster spanning multiple Kubernetes clusters. Imagine a logical control plane that orchestrates multiple Kubernetes masters similar to how each master controls the nodes within its own cluster.

In this tutorial, we will configure a federated cluster that spans Kubernetes clusters running in three continents — Asia, Europe, and America.

When combined with a global ingress, traffic can be automatically routed to the nearest cluster. If the application health check fails in any specific cluster, the request will be automatically forwarded to the next available cluster.

It is also possible to federate clusters running in different environments including public cloud and on-premises data center. But, to keep it simple, we will stick to Google Cloud Platform for this guide.

To complete the tutorial, you need an Ubuntu box with Google Cloud SDK and kubectl tool installed. Of course, you also need to have an active account with GCP to deploy resources. If you have a custom domain, update the DNS settings to point it to Google Cloud DNS Name Servers.

Let’s start by creating a zone for the domain in Google Cloud DNS. This will be used by the federated control plane for cross-cluster service discovery.

Now, let’s go ahead and create three Kubernetes clusters in Asia, Europe, and America.

The above commands create a cluster, points kubectl to it, and then adds the GCP user to the cluster-admin role. Let’s repeat these steps to create the remaining two clusters.

Checking the GCP Console will show all the three clusters up and running.

Since we will be switching the cluster contexts, it makes sense to rename the entries in local kubeconfig. By default, GKE names the context based on the GCP project id, cluster id, and the zone, which makes it cumbersome to use.

Running the following commands will rename default GKE cluster context to more representative names.

Don’t forget to replace janakiramm-sandbox with your own GCP project id. Let’s check the latest contexts in kubeconfig file by running kubectl config get-contexts. You should see shorter names for each context.

I strongly encourage you to explore the structure of config file available at $HOME/.kube location.

We now have everything in place to create a federated cluster. For this step, you need to download the kubefed CLI, which runs only in Linux at this time.

This step is the most crucial since it creates the federated control plane. After a few minutes, you should see the below output.

A federated control plane has been created in the GKE cluster deployed in US Central. The local kubeconfig is also updated. The API endpoint for both the CLIs — kubectl and kubefed — is available at

If we visit the Cloud Load Balancer section of GCP Console, we will notice a new load balancer there. Since the federation is hosted by the cluster deployed in us-central1-a, the load balancer is also provisioned in the same cluster.

When the request is sent to the control plane, it goes via the load balancer to one of the nodes that responds to the API.

Let’s go ahead and join all the three clusters to the federated control plane.

It’s time for us to check if all the clusters are successfully registered with the federated control plane.

Due to a bug in kubefed, the default namespace is not present in the federated control plane. We can create it with the below command.

We are now done with all the steps to create the federation. The next step is to deploy a workload and test it, which we will continue in the second part of this tutorial.

I will introduce an open source tool from Google called kubemci that can configure a multicluster ingress. Using that, we will be able to expose the distributed workload through a single IP address. Stay tuned for the second and the final part.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.