From Containers to Container Orchestration

May 11th, 2016 7:49am by Janakiram MSV

Janakiram MSV is the Principal Analyst at Janakiram & Associates and an adjunct faculty member at the International Institute of Information Technology. He is also a Google Qualified Cloud Developer, an Amazon Certified Solution Architect, an Amazon Certified Developer, an Amazon Certified SysOps Administrator, and a Microsoft Certified Azure Professional. His previous experience includes Microsoft, AWS, Gigaom Research, and Alcatel-Lucent.

The Docker platform and surrounding ecosystem contain many tools to manage the lifecycle of a container. Just one example, Docker Command Line Interface (CLI) supports the following container activities:

Pulling a repository from the registry.
Running the container and optionally attaching a terminal to it.
Committing the container to a new image.
Uploading the image to the registry.
Terminating a running container.

While the CLI meets the needs of managing one container on one host, it falls short when it comes to managing multiple containers deployed on multiple hosts. To go beyond the management of individual containers, we must turn to orchestration tools.

Orchestration tools extend lifecycle management capabilities to complex, multi-container workloads deployed on a cluster of machines.

By abstracting the host infrastructure, orchestration tools allow users to treat the entire cluster as a single deployment target.

Baseline Features

The process of orchestration typically involves tooling that can automate all aspects of application management from initial placement, scheduling and deployment to steady-state activities such as update, deployment, update and health monitoring functions that support scaling and failover. These capabilities have come to characterize some of the core features users expectations offer modern container orchestration tools.

Declarative Configuration

Orchestration tools provide an option for DevOps teams to declare the blueprint for an application workload and its configuration in a standard schema, using languages such as YAML or JSON. These definitions also carry crucial information about the repositories, networking (ports), storage (volumes) and logs that support the workload. This approach allows orchestration tools to apply the same configuration multiple times and always yield the same result on the target system. It also allows the tools to accept different configurations for the same application during the various stages of development, testing and production for different target environments.

Rules and Constraints

Workloads often have special policies or requirements for host placement, performance and high availability. For example, it’s pointless to provision the master and slave database container on the same host; it defeats the purpose. Similarly, it may be a good idea to place in-memory cache on the same host as the web server. Orchestration tools support mechanisms for defining the affinity and constraints of container placement.

Provisioning

Provisioning, or scheduling, deals with negotiating the placement of containers within the cluster and launching them. This process involves selecting an appropriate host based on the configuration. Apart from a container-provisioning API, orchestration tools will invoke the infrastructure APIs specific to the host environment.

Discovery

In a distributed deployment consisting of containers running on multiple hosts, container discovery becomes critical. Web servers need to dynamically discover the database servers, and load balancers need to discover and register web servers. Orchestration tools provide, or expect, a distributed key-value store, a lightweight DNS or some other mechanism to enable the discovery of containers.

Health Monitoring

Since orchestration tools are aware of the desired configuration of the system, they are uniquely able to track and monitor the health of the system’s containers and hosts. In the event of host failure, the tools can relocate the container. Similarly, when a container crashes, orchestration tools can launch a replacement. Orchestration tools ensure that the deployment always matches the desired state declared by the developer or operator.

A Closer Look at Three Popular Orchestration Platforms

Docker Swarm

The objective of Docker Swarm is to use the same Docker API that works with the core Docker Engine. Instead of targeting an API endpoint representing one Docker Engine, Swarm transparently deals with an endpoint associated with a pool of Docker Engines. The key advantage to this approach is that the existing tools and APIs will continue to work with a cluster in the same way they work with a single instance. Docker’s tooling/CLI and Compose are how developers create their applications, and therefore, they don’t have to be recoded to accommodate an orchestrator.

Docker Swarm comes with several built-in scheduling strategies, giving users the ability to guide container placement so as to maximize or minimize the spread of containers across the cluster. Random placement is supported as well.

Docker seeks to follow the principle of “batteries included but removable,” meaning that while it currently ships with only a handful of simple scheduling backends, in the future it may support additional backends through a pluggable interface. Based on the scale and complexity of a given use case, developers or operations staff might choose to plug in an appropriate alternative backend.

Docker Swarm supports constraints and affinities to determine the placement of containers on specific hosts. Constraints define requirements to select a subset of nodes that should be considered for scheduling. They can be based on attributes like storage type, geographic location, environment and kernel version. Affinity defines requirements to collocate containers on hosts.

For discovering containers on each host, Swarm uses a pluggable backend architecture that works with a simple hosted discovery service, static files, lists of IPs, etcd, Consul and ZooKeeper.

Swarm supports basic health monitoring, which prevents provisioning containers on faulty hosts.

Kubernetes

Coming from Google — a company that claims to deal with two billion containers every day — Kubernetes enjoys unique credibility.

Kubernetes’ architecture is based on a master server with multiple minions. The command line tool, called kubecfg, connects to the API endpoint of the master to manage and orchestrate the minions. Below is the definition of each component that runs within the Kubernetes environment:

Master: The server that runs the Kubernetes management processes, including the API service, replication controller and scheduler.
Minion: The host that runs the kubelet service and the Docker Engine. Minions receive commands from the master.
Kubelet: The node-level manager in Kubernetes; it runs on a minion.
Pod: The collection of containers deployed on the same minion.
Replication controller: Defines the number of pods or containers that need to be running.
Service: A definition that allows the discovery of services/ports published by each container, along with the external proxy used for communications.
Kubecfg: The command line interface that talks to the master to manage a Kubernetes deployment.

The service definition, along with the rules and constraints, is described in a JSON file. For service discovery, Kubernetes provides a stable IP address and DNS name that corresponds to a dynamic set of pods. When a container running in a Kubernetes pod connects to this address, the connection is forwarded by a local agent (called the kube-proxy) running on the source machine to one of the corresponding backend containers.

[cycloneslider id=”ebook-3-sponsors”]

Kubernetes supports user-implemented application health checks. These checks are performed by the kubelet running on each minion to ensure that the application is operating correctly. Currently, Kubernetes supports three types of health checks:

HTTP health check: The kubelet will call a web endpoint. If the response code is between 200 and 399, it is considered a success.
Container exec: The kubelet will execute a command within the container. If it returns “OK,” it is considered a success.
TCP socket: The kubelet will attempt to open a socket to the container and establish a connection. If the connection is made, it is considered healthy.

Apache Mesos

Apache Mesos is an open source cluster manager that simplifies the complexity of running tasks on a shared pool of servers. Originally designed to support high-performance computing workloads, Mesos added support for Docker in the 0.20.0 release.

A typical Mesos cluster consists of one or more servers running the mesos-master and a cluster of servers running the mesos-slave component. Each slave is registered with the master to offer resources. The master interacts with deployed frameworks to delegate tasks to slaves. Below is an overview of Mesos’ architecture:

Master daemon: The mesos-master service runs on a master node and manages slave daemons.
Slave daemon: The mesos-slave service runs on each slave node to run tasks that belong to a framework.
Framework: An application definition consisting of a scheduler that registers with the master to receive resource offers, along with one or more executors to launch tasks on the slaves.
Offer: The list of a slave node’s resources. Each slave node sends offers to the master, and the master provides offers to registered application frameworks.
Task: The unit of work scheduled by a framework to be executed on a slave node.
Apache ZooKeeper: The software used to coordinate the collection of master nodes.

Unlike other tools, Mesos ensures high availability of the master nodes using Apache ZooKeeper, which replicates the masters to form a quorum. A high availability deployment requires at least three master nodes. All nodes in the system, including masters and slaves, communicate with ZooKeeper to determine which master is the current leading master. The leader performs health checks on all the slaves and proactively deactivates any that fail.

When Mesos is used in conjunction with Marathon, service discovery can be enabled based on the HAProxy TCP/HTTP load balancer, along with an assistant script that uses Marathon’s REST API to regenerate a HAProxy configuration file periodically. Alternatively, Mesos-DNS, a DNS-based service discovery mechanism, has recently been released in beta.

Summary

The container ecosystem is growing rapidly. From major infrastructure companies to PaaS vendors to early-stage startups and even in serverless computing, everyone is clamoring to stake out their place in the ecosystem. There are many contributors working on container orchestration tools, as these are essential for deploying real-world applications, thus driving the adoption of Docker and containers. We attempted to highlight some of the key contributors building orchestration tools, but there is more to it than just explicit orchestration tools — it’s also important to look at the build, deployment, CI/CD, PaaS, and other tools that orchestrators interact with, which we cover at great length in the Automation and Orchestration Directory.