What news from AWS re:Invent last week will have the most impact on you?
Amazon Q, an AI chatbot for explaining how AWS works.
Super-fast S3 Express storage.
New Graviton 4 processor instances.
Emily Freeman leaving AWS.
I don't use AWS, so none of this will affect me.
Cloud Native Ecosystem / Cloud Services / Kubernetes

Tutorial: GitOps in Multicluster Environments with Anthos Config Management

This month, analyst Janakiram MSV launches an exclusive series on The New Stack, about Google Cloud Platform’s Kubernetes service, Anthos. Each part of the series focuses on a specific aspect of Anthos. The series covers multi-cluster registration, Anthos configuration management, and launching “click to deploy” applications from the GCP Marketplace.
Jun 19th, 2020 9:57am by
Featued image for: Tutorial: GitOps in Multicluster Environments with Anthos Config Management
Feature image via Pixabay.

In the third part of this series, we will use GitOps-style deployment to push workloads across all registered clusters through Anthos Config Management (ACM).

GitOps encourages maintaining configuration-as-code and environment-as-code in a central source code repository. This gives us a chance to version control configuration and environments along with source code.

Since Kubernetes uses YAML or JSON files for specifications, it becomes easy to combine these artifacts with code.

Google built a tool called Config Sync which acts as the bridge between an external source code repository and the Kubernetes API server. Anthos Config Management is based on Config Sync to extend it to multicluster scenarios.

In this tutorial, we will use a GitHub repository that acts as a single source of truth for deployments and configuration. A component of ACM is installed into each of the registered Anthos clusters to monitor the external repository for any changes and synchronizing them with the cluster.

ACM ensures that all the clusters have the same state as defined by the specifications in the repository.

ACM supports a structured or unstructured repository for configuration. A structured repo will have a hierarchy for namespaces and cluster-wide resources. An unstructured repo can be used to maintain ad hoc configuration from one or more YAML file consisting of multiple Kubernetes objects. Unstructured repositories are helpful when you want to expand Helm charts and apply it via ACM.

Let’s get started with the steps to implement GitOps with ACM.

Installing the Configuration Management Operator

The Config Management Operator is a controller that manages the installation of Anthos Config Management in a Kubernetes cluster.

We need to install this operator in all the three clusters — GKE, EKS, and AKS.

Download the YAML spec for the operator from Google Cloud Storage and apply this to each cluster.

Make sure you run this command on all the clusters.

Verify the creation of the operator with the below command:

Configuring Clusters for ACM

Google ships a nifty utility called nomos that can be used to manage ACM. Download and add it to the path. The below commands work on macOS.

Now is the time to connect the GitHub repository with ACM. We will use a sample repo from Google Cloud documentation. Feel free to explore the structure of the repo.

Since we are using a public repo, we can set the secretType to none.

Create the below YAML file for each cluster by replacing the clusterName with the registered clustered name in Anthos.

To get the names of registered clusters, run the below command:

Apply the below spec on each cluster. Make sure that the clusterName is accurate.

If the command succeeds, Kubernetes updates the Config Management Operator on each cluster to begin syncing the cluster’s configuration from the repository. To verify that the Config Management Operator is running, list all Pods running in the config-management-system namespace:

The pods listed above are responsible for monitoring the repo and applying the changes to the Kubernetes API server.

We can use the nomos utility to check the status of the clusters. Run the below command to see if all the clusters are in sync.

Exploring the Clusters and Repo

The foo-corp repo includes configs in the cluster/ and namespaces/ directories. These configs are applied as soon as the Config Management Operator is configured to read from the repo.

All objects managed by Anthos Config Management have the label set to

List namespaces managed by Anthos Config Management:

Switch the context to each cluster and check the namespaces. All of them have the same configuration.

The namespaces shown here are defined in the GitHub repo under the foo-corp/namespaces folder.

We will test what happens when one of the clusters deviates from the configuration specified in the repo.

Let’s delete the audit namespace in one of the registered clusters.

The namespace gets deleted immediately, but within a few seconds, it becomes visible.


With ACM, Anthos can ensure that each registered cluster has the desired state of the configuration by constantly syncing it with the ACM repo. In many ways, this mimics the same workflow as the Kubernetes controller maintaining the desired number of replica sets at any given time.

Anthos Config Manager is one of the core building blocks of Anthos that enables centralized configuration of cluster states.

In the next part of this series, we will explore how to configure Amazon EKS cluster for deploying “click to deploy” Kubernetes apps from the GCP Marketplace. Stay tuned!

Janakiram MSV’s Webinar series, “Machine Intelligence and Modern Infrastructure (MI2)” offers informative and insightful sessions covering cutting-edge technologies. Sign up for the upcoming MI2 webinar at

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.