Docker Launches LinuxKit and the Moby Project for Building Containerized Infrastructure
Docker wants to make containers, rather operating systems, the base building block for data center operations. The company has unveiled two new open source projects for system builders to create their own modular container-based systems, using both Docker and third-party components.
“In order for us to leverage all the innovation that we as an ecosystem are putting together, we need to start collaborating on the assembly level,” said Patrick Chanezon, Docker IT lead, in an interview with The New Stack. The company is revealing these tools at its annual user conference, DockerCon, being held in Austin this week.
One project, called LinuxKit, is a toolkit for assembling operating systems for containers. The second endeavor, the Moby Project, sets the stage for collaborating on interchangeable components, allowing third parties to build container-based systems from a component library and reference blueprints.
LinuxKit provides a container-based approach to building a customized, slimmed down Linux subsystem for each type of container, which could prove useful for system builders that want to assemble their own select Linux features for specialized hardware or features. Based on containerd, each LinuxKit subsystem would have its own Linux kernel, and each system daemon, or system service, gets its own container.
The subsystem, once packaged as an ISO image, can boot on bare metal, or in a virtualized environment. Docker will maintain all the Linux subsystem components it uses as part of its own Docker offerings.
“Linux is a component of Docker” — Patrick Chanezon
Containers could provide a natural and flexible base for running all data center operations, as they would allow for easy assembly of components, as well as minimize the dependence on operating systems, with all the attendant security flaws. By pairing back the OS to only the needed components, Docker itself has trimmed its own container OSes to a relatively petite 35Mb to 100Mb range.
“Many of our customers have asked us to go beyond Linux,” Chanezon said. “Linux is a component of Docker. With LinuxKit, we let you assemble that component in a way that is secure and portable.”
Docker itself has already used the technologies in LinuxKit to build customized OSes for both the Docker Desktop and Cloud Enterprise Edition SKUs. For this work, Docker has partnered with the Linux Foundation and has gotten input from ARM, IBM, Intel and Microsoft.
Intel, for instance, could LinuxKit to build container-based system on top of its own chip-secured Clear Containers technology. HPE could use the technology build customized containerized systems for its own hardware.
Call Me Ishmael
The Moby Project aims to bring some order to the increasingly unwieldy process of assembling-based systems, usually from multiple components both from Docker and third-party providers.
Moby “is the most important open source project that Docker has [launched] since the original Docker,” Chanezon said.
The company started componentizing parts of its system, such as Notary, around 2014, work that led to Moby. Docker developers “needed to assemble components in different configurations, so we are always trying to leverage or reuse whatever we have,” Chanezon said.
Moby will provide a framework and common collaborative area of sorts for container vendors and service provider to organize and a common set of tooling and best practices for building and augmenting container-based systems, covering all aspects such as monitoring, orchestration, networking and the like.
Thus far, Docker already has a library of more than 80 containerized components, including Swarm, containerd, Docker Build and even LinuxKit. Many third-party components are now hard-wired into Linux distributions. Docker is welcoming more contributions from the community.
A builder of Internet-of-Things devices may need a vastly different set of components than a company building data center systems. With Moby, both can share common components and also enjoy the flexibility of wiring their own components into a common assembly.
“There are a lot of people who are coming into the container ecosystem that go beyond running cloud-native application on the cloud: IoT, desktop, and lots of different environments,” he said.
Docker is in the process of packaging all its own components as containers, and will also offer a reference assembly, one being based on Docker Community Edition. The Moby tool itself can be used to assemble projects from different components.
Taken together, these projects point to treating Docker containers as part of a larger assemblage of technologies for running containerized applications, one that relies as much on third-party providers as much as on Docker itself. Having a common base for building systems can help system builders accelerate their efforts in building out new capabilities, Docker argues.
In its latest survey of 10,000 users, monitoring company Datadog found that that 18.8 percent had adopted Docker, up 40 percent from a year ago when it was 13.6 percent. Perhaps even more surprising is that once onboard, Docker users, on average, nearly quintuple the number of containers they run within production within 10 months. And this success spreads to third-party providers of Docker tools. Datadog also found that 40 percent of Docker users are deploying an external orchestrator, such as Kubernetes or Amazon ECS.
Thus far Docker components — including containerd, Swarm, libcontainer and others — have been downloaded more than 12 billion times, according to the company.
