Supercharge CoreDNS with Cluster Addons
Infoblox sponsored this post.
There has been an increasing demand from users to be able to manage the health, status, rollout, rollback, etc., of CoreDNS in a Kubernetes cluster; and not just rely on CoreDNS being managed by the cluster management tools. Since the use of Operators in Kubernetes is now generally accepted, the aim of the cluster-addons project is to bring the benefit of Operators to all addons.
Addons extend the functionality of Kubernetes. Some of the addons, like CoreDNS and kube-proxy, are considered essential to the functionality of a Kubernetes cluster and are shipped along with cluster management tools like Kubeadm, Kops, etc. There are other addons which can be installed as an option and are not as critical, but help users manage their clusters — e.g., Calico, Kubernetes Dashboard, and NodelocalDNS.
The purpose of the cluster-addons project is to enable you to manage addons in a better way, with the help of Operators.
An Operator is a method of packaging, deploying and managing a Kubernetes application.
Each Operator will have its own CustomResourceDefinition (CRD), where users will be able to manage the addons — in terms of installing, upgrading, choosing the version, etc. — by modifying the CRD without being dependent on the cluster management tools.
The Need for Cluster-Addon Operators
Today, cluster management tools like Kubeadm, Kops, Cluster API, etc., are bundled with essential addons like CoreDNS and kube-proxy.
Some of the challenges faced are:
- Users want to have more flexibility when managing the addons installed in their clusters. Currently, addons get automatically managed alongside the lifecycle of the cluster. For example, if a user upgrades or downgrades their version of Kubernetes, the version of the addon(s) gets upgraded or downgraded along with it.
- It is becoming difficult to manage cluster management tools, due to the addons becoming increasingly complex.
In the past, there was an attempt to solve these problems via the bash version of addon-manager. However, this was unsuccessful and hasn’t been widely adopted.
The cluster-addons project focuses on solving these challenges by introducing Operators. Operators allow users to decouple the lifecycle of the addons from the lifecycle of the cluster.
Each Operator is defined by a CRD. Users will be able to tailor the addon in a limited way (e.g., installing, upgrading) through the CRD. Each Operator has its own Controller. A Controller is a control loop that watches the state of your addon resources and makes or requests changes where needed. The controller tries to move the current state of the addon resources closer to the desired state.
The CoreDNS Operator
CoreDNS has been the default DNS addon since Kubernetes v1.11 and is one of the most essential addons for having a functional Kubernetes cluster.
The CoreDNS Operator has been specifically highlighted here, as CoreDNS is one of the most complicated addons. It is the first Operator to be implemented to showcase the power of the Operator and its capability of making CoreDNS easier to manage on a cluster.
As part of its basic functionality, the CoreDNS Operator is capable of installing CoreDNS, upgrading the CoreDNS version, and migrating the Corefile automatically during upgrade, to ensure it is up to date and compatible with any version of CoreDNS — providing the user with a seamless experience.
The Operator continuously monitors the CoreDNS resources (Deployment, ConfigMap, Service, etc.) with the help of a controller.
The CoreDNS specifications can be modified through the CustomResource (CR). An example CoreDNS CR is as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
apiVersion: addons.x-k8s.io/v1alpha1 kind: CoreDNS metadata: name: coredns-operator namespace: kube-system spec: version: 1.7.0 dnsDomain: cluster.local dnsIP: 10.96.0.10 corefile: | .:53 { errors health { lameduck 5s } ready kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure fallthrough in-addr.arpa ip6.arpa ttl 30 } prometheus :9153 forward . /etc/resolv.conf { max_concurrent 1000 } cache 30 loop reload loadbalance } |
As seen above, the CoreDNS Version, DNS Domain, DNS IP and Corefile can be modified by modifying the CR spec. The Controller will register the changes applied to the CR and will reconcile with the CoreDNS resources to reflect the changes.
To learn how to install the CoreDNS Operator on your cluster, you can follow the instructions here.
Creating Your Own Operator
The cluster-addon operators are created with the help of Kubebuilder, a framework for creating your own Kubernetes APIs using CRDs. If you are interested in creating your own Operator, please check out these step-by-step instructions on how to create and deploy your Operator.
Status of the Project
Today, the cluster-addon repository contains many Operators — such as CoreDNS, NodelocalDNS, kube-proxy, etc. — which can be installed and enable users to manage these addons on their Kubernetes clusters.
The cluster-addon project also contains the addon-installer library, which functions as a small implementation for applying addon operators to the Kubernetes cluster using kubectl and Kustomize.
Future Work
We are working towards integrating the Operators into cluster management tools such as Kubeadm, Kops and Cluster API.
Reaching Out
If you are interested in the cluster-addons project, want to explore Operators, want to discuss addons or new features, or you need help, you can contact us at #cluster-addons Slack and the SIG Cluster Lifecycle group.
Feature image via Pixabay.