OpenStack Summit: IBM Demonstrates Live Migration of Docker Containers
Two IBM software engineers have demonstrated the ability to move a live Docker container from one host to another, without interrupting any service that the container is running.
Although Docker will probably not tout live migration as a feature for its upcoming Docker 1.13 release, this version of the Docker Engine should be able to support this capability, initially in a limited way, thanks to a new capability called Checkpoint and Restore, said Phil Estes, senior technical leader for IBM’s Open Cloud Technologies.
Estes, along with IBM engineering colleague Shaun Murakami, an IBM software engineer, demonstrated this capability at the OpenStack Summit, being held in Barcelona this week.
Checkpoint and Restore will allow a user to freeze a running container by checkpointing it, which involves capturing the container’s current state into a collection of files on disk, so it can be later restored from the point it was frozen.
Docker plans to offer checkpoint and restore as an experimental feature in the next release of the Docker engine, offering it as a way of making snapshots of containers. Live migration of containers could make it easier for administrators to move containers around their systems without disrupting services, adding to the agility and robustness of such services.
The Open Container Initiative‘s runC reference container engine also supports CRIU.
To offer this capability, Docker uses an external tool called CRIU (Checkpoint/Restore In Userspace), created by Virtuozzo, a spin-off of desktop virtualization software provider Parallels.
CRIU captures all aspects of the process state, including thread information, user and group identifiers (UID/GID), memory state, open files, TCP state, timer signals, and network sockets. “This becomes a set of metadata that I can take somewhere else and restore,” Estes explained.
CRIU was originally applied to the task of migrating live virtual machines, and has had kernel support since Linux 3.11 (though much of CRIU’s work is done in the userspace). So extending its use to containers shouldn’t, in theory, be too difficult. It would be a natural extension to have CRIU do the same checkpoint and restore, allowing Docker to restart the container in another location from within the Docker engine.
“Containers are simply processes on your Linux system,” Estes said.
Murakami and Estes compiled their own version of the Docker engine, one that merged upstream checkpoint and restore patch.
Murakami ran three demonstrations of the live migration. In the first demo, he moved a counter holding a simple counting app, from host to another, as seen here in this short Twitter clip:
IBM's @stmuraka demos a live migration of a container across two hosts, using @__criu__ and a patch to the Docker engine #OpenStackSummit pic.twitter.com/yTyTlZ9G0Z
— The New Stack (@thenewstack) October 25, 2016
When the containerized app was moved to the new host, the counter picked up where it left off, rather than returning to zero, showing how the app, in effect, was not interrupted by the move.
In the second demonstration, Murakami was able to move from one host to another a more sophisticated application, based on a live Redis in memory data store that reported in real-time the number of mentions on Twitter, of #OpenStack and #Docker. Again, the move did not interrupt the flow of data from the application.
The third, most ambitious, demonstration didn’t quite pan out, showing the additional work needed to be done before Docker, or anyone else, could offer live migration as a production-ready feature. This demo involved keeping a live VLC-based video stream running through a host change. Issues around socket reassignment seemed to bring the video stream to a halt when the VLC software was assigned to a new host.
Admittedly, moving a live video stream service from one host to another is the most challenging of live migration tasks, but such a capability would be of great interest to high-volume video services. Again the edge cases need to be hammered out, Estes warned.
There are two different approaches to moving a container, Murakami explained. Containers with stateless workloads, meaning no ties to persistent data sources, are the easiest to move: The second host can merely pull, or even pre-fetch, an identical image and drop in the checkpoint-metadata.
Those with persistent data have additional challenges, Murakami said, as file system changes also need to be conveyed to the new host.
