Docker’s Unikernel Experiment Begins Paying Off with Mac OS Libraries
The promise of the unikernel is that it would optimize the way containers are packaged, eliminating even more redundant code by replacing full-featured operating system kernels with libraries containing just the code that packaged apps may use. Last January, Docker purchased Unikernel Systems, one of the first companies in this emerging space, and immediately began putting to rest the now clearly mistaken notion that unikernel packaging for workloads would replace Docker and CoreOS containers.
On Wednesday, the first day of O’Reilly’s OSCON 2016 conference in Austin, Texas, Docker Inc. revealed in a company blog post how its engineers have been working to implement unikernels to simplify container-based development on Mac OS X.
With these projects ready for public consumption, Docker Inc. is now open-sourcing a trio of unikernel-based “kit” projects including — perhaps most ironically of all — a lightweight hypervisor called HyperKit that purports to enable any Mac application at all to enable virtualization, even without a need to escalate privileges first.
“HyperKit applications can take advantage of hardware virtualization to run VMs, but without requiring elevated privileges or complex management tool stacks,” wrote Anil Madhavapeddy, who led Unikernel Systems during the startup phase, and who now officially works as a Docker Inc. engineer. “Since HyperKit is broadly structured as a library, linking it against unikernel libraries is straightforward. For example, we added persistent block device support that uses the MirageOS QCow libraries written in OCaml.”
During his time on stage at OSCON, Docker founder, and CTO Solomon Hykes personally released the repository containing HyperKit, the filesystem interface unikernel DataKit, and the Ethernet-to-socket networking library VPNKit, to GitHub for public download, distribution, and contribution.
Madhavapeddy spoke in further detail about Wednesday’s release to The New Stack’s Alex Williams, for an upcoming podcast produced Wednesday at OSCON.
The Beauty of Immutability
It was during a presentation last July that Madhavapeddy’s case for unikernels and what he described as their inherent immutability, first came together in a public forum. At a programming language workshop called NetPL in Prague, Czechoslovakia, he told fellow language makers, “We’re building layers and layers of software, dropping them onto the cloud, with more and more layers of emulation, and the entire software stack is starting to fall apart — security holes; very, very hard to program; hard to manipulate in an interesting, programmatic way.”
It didn’t make sense to Madhavapeddy that so much effort would be expended to producing beautifully functional, sleek, streamlined applications, just to have them dropped onto twenty million or so lines of operating system code, in a manner that he described as inherently unsafe. Docker Inc.’s timely adoption of unikernels as a packaging and deployment methodology may have been a dodging of a very disruptive bullet.
“Because we’re statically linking only the required libraries, where we are removing a lot of unnecessary services and lots of unused code from the finished artifact,” explained Amir Chaudhry, another member of the Unikernels team, during a Docker Inc. presentation a few months ago, “this also helps us increase the speed, because those finished artifacts are much, much smaller.” Chaudhry went on to explain how the library to which he contributes, called MirageOS, can be booted during the time consumed in the setup of a TCP connection.
The End of OS Vulnerabilities?
It’s that fact, and others like it, that radically alter the potential role of the functions traditionally performed by operating systems and even by hypervisors. We’re accustomed to thinking of the OS as a massive pile of libraries that rests in memory, most of which is waiting never to be used — not unlike a fact-checker for a political speechwriter. To facilitate backward compatibility and support for “legacy” software and systems, OSes have had to build layer upon layer of encrusted support code.
Debugging these layers is not at all unlike a geological expedition, complete with the need for advanced archaeology to sort through decades of untouched logic. It is within these layers of old code that the world’s most common, and most effective, vulnerability exploits typically lurk.
"No one ever came to us and said 'We love VirtualBox'" @justincormack on the origins of @Docker for Mac #Dockercon pic.twitter.com/oFTVtpIL2U
— The New Stack (@thenewstack) June 20, 2016
We could continue to develop faster and more heavily crowdsourced methods to spot these defects and, perhaps, correct them. Or, as the Unikernels team has demonstrated, we could introduce a substantially reduced stack where these layers simply do not exist. You wouldn’t need a sophisticated container scanner to poke through layers of code that are no longer there.
Docker’s integration of unikernel architecture is already solving potentially major architectural pitfalls for the company before it ever had to acknowledge they existed. Its hypervisor kit for Mac OS has already been put to use, as part of the company’s Docker for Mac developer toolset. It is this already battle-tested functionality that the company open-sourced through GitHub on Wednesday.
At OSCON on Wednesday, DeferPanic founder Ian Eyberg presented a session that made the case that “unikernels will overtake the container ecosystem within the next few years.” That may not be how Docker Inc. sees it, especially after giving a pretty convincing demonstration Wednesday of how the two have already integrated.
Docker is a sponsor of The New Stack.
Feature Image via Pixabay.
