The Docker Workaround: How ProfitBricks Did It
Docker. It’s as if you cannot escape reading that name or hearing about the wonders of containers. The question is becoming common and the demand acute within the infrastructure hosting space: what are solutions that users can easily utilize when it comes to deploying and managing Docker environments?
The Wild World of the Docker Ecosystem in 2015
Some early issues we faced when evaluating how to approach building a Docker delivery and clustering platform involved looking at the various competing technologies on the market: Apache Mesos, Google Kubernetes and the various components in Docker itself. It was, and still is, difficult to untangle these technologies into domains that represent their core functionality versus what might be considered edge, and to determine where overlap occurs and where it doesn’t. It was also apparent that many projects were just too new, e.g. Kubernetes; while other projects, such as Mesos, have only recently started supporting Docker (Mesos 0.20.1).
You want to simply bring up a Docker environment, span machines, deploy containers, manage volumes, and scale. To ensure we could meet that requirement we stripped away everything except the ProfitBricks infrastructure, its RESTful API, Docker and the companion components: Machine and Swarm. We didn’t want to contend with introducing too much complexity too early into the development and learning phase as we built out the Docker hosting platform.
While going with pure Docker reduced complexity, it ramped up the number of issues we dealt with during development and still contend with as we actively work through our own bugs.
Nothing is more frustrating than refactoring or burning cycles on issues from other, active projects. But, this is also what we signed up for: we needed to build a platform customers could spin-up, manage and span their containers across.
The Elephant in Everyone’s Room: Docker Security
Security is a common and absolute requirement, so when we configure the Docker daemon we bind a certificate to it; however, we also have a common scenario where we must revoke that same, self-signed certificate. The issue we encountered is that in order for you to revoke a certificate and have the Docker daemon respect that revocation, you are required to re-configure on localhost and restart the daemon, which isn’t ideal. Additionally, there is no API for these types of management tasks, requiring us to automate shelling into the system to make the change and then exiting. For the moment, there is no workaround — at least, we weren’t able find one that would address the issue in the way we would have hoped. This leaves us with an ad-hoc process that shells, reconfigures and restarts.
Docker Images — Not as Defined as We had All Hoped
A gotcha we ran into that we can only imagine others have encountered is the relative opaqueness of a pre-existing image and the requirements the container has for it to run successfully. An aspect of our application pulls a list of images and their corresponding tags from DockerHub. We then pre-load the config into the interface.
We started to run into this gotcha with some containers requiring environment variables, links, or volumes to be created at run time, yet very little information on those requirements being there in the first place (except in the Dockerfile if present) and no real way to automatically suck them in. While this isn’t a direct Docker issue, it does cause headaches and, at times, bewilderment as to why a container just isn’t spinning up, or has inherited an environment variable that we were expecting to be blank. We also saw issues where some pre-built containers — even those from official repositories such as MySQL — set default values for optional environment variables in the container, yet the user is unaware as to what is optional, what is required, and what has a predefined value that probably should be changed, e.g. password handling in MySQL.
Autoscaling and Persistent Volumes — Now Available for Docker Containers
To overcome the lack of autoscaling in Docker, we developed our own system for monitoring the health of the underlying host system to determine if it’s close to resource exhaustion or not. When container density increases, we bump up core and memory. When storage requirements increase, we resize the volume. Set limits and the agent respects them. So on so forth, ensuring you always have a healthy Docker environment.
One of the other challenges we found with building a Docker platform is how to handle persistent volumes. Docker is ephemeral in nature, but applications, such as databases, often require data to persist across system restart. Sure, let’s lose state, but I don’t want to lose my PostgreSQL data. Again, here we combine functionality and allow users to add up to eight volumes with up to 2TB capacity. These volumes can also be moved between hosts.
What’s Next for the World of Docker Hosting?
We’re still working through multiple issues — both our own and those imposed upon us by the different Docker projects we rely upon — but we’re bullish on the transformative power Docker on the container ecosystem and cloud computing space, and are well positioned for it. As we iterate through improving the platform, adding features such as Swarm and other things, we will continue to share our growing pains and successes with Docker in our community.
Achim Weiss is chief executive officer and co-founder of ProfitBricks. In 1995, Achim Weiss broke off his studies to co-found the web hosting company “Schlund+Partner.” The company “Schlund+Partner” later became the 1&1 web hosting product line. From 1995 to 2008, he was technical director for several projects: the largest web hosting platform in the world, the second largest DSL platform, a video on-demand delivery network, the largest email backend in Europe, and a universal billing system. In 2010 he co-founded ProfitBricks, a cloud computing IaaS provider that today is one of the leading providers. ProfitBricks offers a painless cloud experience for all IT users with no learning curve, and boasts flexible cloud servers and networking, an integrated Data Center Designer tool for visual control over the cloud and the best price/performance value available.
ProfitBricks is a sponsor of The New Stack.
Feature image via Flickr Creative Commons.
