Containerd 1.0 Release Becomes the Public Face of Containers
There may be a plurality of operative components inside an OCI standard container, though for now, two are of prime importance. The runccomponent is the executive — the part which makes a container functional unto itself. The second part of the puzzle, containerd acts as the part that “supervises” the lifecycle of containers, and that communicates with the outside world via API calls.
That functionality may replace the need for the continual presence of a full container engine in a production system, clearing the way for Kubernetes, Mesosphere DC/OS, and other container orchestration engines.
“With the next version of the Docker platform, developers can build and test apps destined for production directly on Kubernetes, on their workstation,” announced Docker Inc.’s Solomon Hykes last October (Hykes has since moved on from chief technology officer into the role of vice chairman). “And ops can get all the benefits of Docker Enterprise Edition — secure multitenancy, image scanning and role-based access control — while running apps in production orchestrated with either Kubernetes or Swarm.”
The Engine That No Longer Connects Us
Although essentially the entire container-driven world has been constructing platforms around containerd-equipped constructs since that time, only in early December 2017 has the project reached the stability level necessary for the Cloud Native Computing Foundation, the project’s steward since last March, to declare it generally available.
Just a few weeks before the release, Docker engineer Stephen Day explained to attendees of DockerCon Europe 17 that containerd makes it feasible for an orchestrator, or a plugin component managed by an orchestrator platform, to effectively manage the entire container lifecycle without needing to include the entirety of Docker Engine or its counterpart. Kubernetes engineers have already demonstrated this principle in action through their use of the orchestrator’s Container Runtime Interface to deliver management functionality — for example, through CRI-O.
Taking over from Docker’s original engine, containerd conducts an interaction with the host operating system to set up primitives for the container’s network namespace and file system, fetch a container’s OCI image from a registry, insert the components from that image into the namespace, and begin execution. It also collects and distributes metrics on the container’s performance for monitoring platforms such as Prometheus, as well as for low-level benchmark drivers such as the independent, open source Bucketbench project.
In a recent webinar, IBM senior technical staff member Phil Estes — the creator of Bucketbench — explained that implementers of containerization required certain functionality that had, up until last year, been locked up in the Docker Engine to be spun off into individual, standardized, interchangeable, and potentially “boring” components. It had been this explicit dependency upon one brand of container engine that had been the focus of contention two years ago, when CoreOS introduced a competing runtime that would have been more loosely coupled with the container structure.
Scuffles such as that one posed the danger of dividing the container development community into camps, based on members’ positions on relatively esoteric architectural tenets.
A Little Less Docker
Since that time, Docker Inc. has enabled the open source initiative behind its technology to spin off just slightly from the commercial organization, to become the Moby Project. In so doing, it set up a contribution structure that effectively deposed Docker from its de facto position as “benevolent dictator for life” (BDFL), and established in its place a technical steering committee (TSC) initially comprised of Docker engineers, but also others including Estes from IBM, and Codeship Director of Engineering Laura Frank.
That committee structure was announced by Estes in a personal blog post last December 13. It’s that structure which he credits with making Moby’s projects more accessible and interesting to a broader body of open source contributors. Although the CNCF is the official steward of containerd, Estes told his webinar last October, the Moby TSC will have influence over its development strategy.
Already, that strategy has led to a less Docker-centric mindset among Moby’s contributors, especially with respect to Kubernetes. With the 1.0 stable release of containerd, developers inserted a few significant features that were not tried during the runtime’s 0.x alpha periods. One such addition is called container extensions, which documentation describes as a means by which components seeking to integrate with the container runtime can store the JSON metadata describing the integration, within a data structure maintained by the container itself.
This would effectively eliminate the need to integrate with a container engine, the way Docker Inc. envisioned two years ago.
“Containerd will be part of that umbrella covered by the TSC,” said IBM’s Estes during his webinar. “A lot of people have been looking for this change for a while, so this is really positive news, and definitely makes containerd and other projects under that Moby umbrella even more open governance-focused, and should be more amenable to a larger community of folks who still wondered about Docker control over some of these projects.
“One of the great aspects of spinning these [components] out, and especially providing them to foundations,” Estes continued, “is really to have them expected to be used beyond the Docker Engine and the Docker product suite.”
Non-Hostile Takeover
In its new role as the outward facing part of Docker containers, explained Estes, containerd will assume many of the responsibilities originally entrusted to the Docker daemon, back when it was a single binary that fused the functions of the build tool, clients, and daemon in one component. It will be the manager or supervisor (the precise language here is still being settled upon) of the runc executive, and it is runc which makes available the intrinsic functions of the container’s internal binary components.
As a container progresses through its lifecycle, containerd makes use of an extensive subsystem to fire off explicit events for each major milestone, such as instantiation, starting, updating, and removal, as represented in the diagram above (courtesy containerd.io). In this diagram, ctr serves as the command line that communicates with the API, although in production, components can make this connection directly instead of just through the command line.
Through an API based on gRPC, routines may be triggered by these events. In the source code of any routine that needs to listen for these events, first a method instantiates a new instance of a container service client. A collection in memory is declared that will contain the queue of returned events. Then, in the simplest formation, a condition-less loop can repeatedly use the .Recv() method to listen for events, then jump to the appropriate response.
What’s most important about this interaction is that it is not between the Docker Engine, or any other container engine, and the client. Rather, it’s the client making contact with a standardized element of the container itself. This way, the orchestrator is no longer vying for contention with the engine for management rights, nor is the orchestrator having to listen for events triggered by the engine that may impact its own management plans one way or the other.
This methodology conceivably alleviates the concerns raised by Google engineer Tim Hockin last February, perhaps foremost among them the long, long chain of components that had to pass events through an interface between the orchestrator and the container engine, like a fire brigade passing water buckets. What’s more, Docker Inc. had been building what Hockin described as “an opinionated stack” of components responsible for lifecycle management. “I don’t want any of that,” he said at the time. “It doesn’t make sense for Kubernetes.”
Now, all that contention may have just become water under the bridge, so to speak. Remarked Estes during his October webinar, “Containerd is a CNCF-based project that could be used by all kinds of people, including VMware, Docker, Kubernetes, IBM Cloud, and Amazon ECS Registry… All these players now have this core container runtime separate from the Docker product suite.”
The Cloud Native Computing Foundation, Codeship and CoreOS are sponsors of The New Stack.
Feature image: A mother and baby beluga whale taken at Vancouver Aquarium by Iwona Erskine-Kellie, released under Creative Commons 2.0.
