Heptio Releases Two New Tools to Make Kubernetes Easier to Manage
Kubernetes, the open-source system for automating the deployment and scaling of containerized applications, does its job really well. It groups an application’s containers into logical units for easy management and discovery, scales all the way from local testing to truly global production, and runs pretty much anywhere.
At the same time, the platform can be intimidating to implement — particularly for new users configuring their first-ever system running on Kubernetes. Kubernetes is quickly becoming the de facto standard for software container orchestration, but accessing it remains a challenge for anyone who is not a hardcore systems engineer.
Heptio, founded by Kubernetes co-creators Joe Beda and Craig McLuckie, is a company that aims to make the platform more accessible. “Our mission is to make Kubernetes more naturally accessible to developers who aren’t necessarily deep back end engineers, and help it become more integrated into the fabric of enterprise IT,” said Heptio CEO McLuckie in an interview.
To that end, Heptio has just announced two new open source projects designed to make Kubernetes easier to use in any production scenario, regardless of where you run your cluster. Heptio Ark is an easy to use disaster recovery tool that both backs up and restores your Kubernetes cluster, while Heptio Sonobuoy is a tool for inspecting cluster state and configuration to ensure everything is correctly configured and working as it should. Both projects are publicly available on GitHub.
All Aboard the Ark
Heptio Ark is named after the literal definition of “ark,” namely, a vessel or sanctuary that serves as protection against extinction. Production use of Kubernetes means managing the challenge of cluster backups and, if disaster strikes, restorations. Mixing in stateful workloads makes things even more complex; pinpointing the correct association between a volume snapshot and its pod(s) at a specific time stamp, for instance, can be tricky.
“Kubernetes is very robust and keeps things alive very actively, but if you have data corruption inside the cluster you need to recover a workload from the backup,” explained McLuckie. “Customers need to be able to well integrate the information inside the cluster with the state that exists outside the clusters.”
Say you run an app that has the system info with it being kept in storage volumes outside the cluster. If something goes awry within a series of apps running in the Kubernetes environment that have state stored outside the Kubernetes envelope, it means sleuthing out what parts of the state are associated with what parts of the storage.
Heptio created the Ark project to solve this issue by integrating the information inside the cluster with the state that exists outside the cluster. “If something bad happens, Heptio Ark makes it simple to recover the workloads that were running, either an entire cluster or a section of a cluster, potentially into a new cluster,” said McLuckie.
And Heptio Ark is not just for critical backup and recovery operations.
“This same disaster recovery mechanism is also a very useful tool for effectively xeroxing out a cluster or a portion of a cluster — extremely useful in testing,” said McLuckie. In short, Heptio Ark acts as a controlled mechanism to replicate one or more segments of the production environment in order to produce a precise replica for QA.
“Being able to more quickly change your apps by very effortlessly creating a reproduction of your production environment means you’re able to experiment — while knowing that, at the end of the day, what you have built will continue to work,” explained McLuckie.
Heptio Ark is being released in an alpha stage in order to solicit feedback. It is currently compatible with three popular public clouds (Amazon Web Services, Azure, and Google Cloud Platform). Since it’s designed to be extensible, Heptio will be adding more environments. You can learn more about the project by checking out the GitHub page.
The View from Below
The company’s other new release is also evocatively named for its function. A sonobuoy is an immersive device that uses sonar to provide a subsurface view of the ocean. Heptio Sonobuoy, said McLuckie, was born from working with customers to resolve issues. “We found ourselves wondering if their installation was correct, and had to spend a lot of time spelunking through their clusters to see what was what.”
Given that there are many different ways to create Kubernetes clusters and many environments used to host them, it’s not always easy to know if a system is properly configured. “We want customers to use the upstream Kubernetes distribution in their own environments, but many struggle to know whether what they’ve built in any way resembles what it should look like,” said McLuckie. “Sonobuoy provides insight into what is happening inside that cluster, so you have much higher levels of awareness and inspectability.”
For example, getting the Kubernetes networking stack set up correctly can be challenging, particularly for newer users. Following the set of installation steps may have the customer believing that the Kubernetes cluster is working — even though it may actually not be connected to outside world via networking. Unfortunately, they only discover this when they try to actually go live.
McLuckie described Sonobuoy as a way for users to inspect their configuration to make sure things are working as intended. “It’s a complex task, so it’s important to know you have done it right,” he said. “That you have in fact established an actual Kubernetes cluster versus something that looks like a Kubernetes cluster, but is not completely functioning as one.”
There was indeed a need for Sonobuoy’s level of inspection. “As an open source community we put a lot of effort into creating great test frameworks, but these are used to qualify Kubernetes test releases to make sure the cluster continues to run as it should,” said McLuckie. “However, these tests are too complex for customers to run.”
At the end of the day, Sonobuoy allows Heptio to provide “a far better, much more nuanced support experience knowing what is happening inside that cluster — which helps us help customers get that much more out of their Kubernetes clusters,”
Running Upstream
Ark and Sonobuoy both exemplify Heptio’s commitment to remain true to Kubernete’s original architecture while allowing customers to be as close as possible to the upstream open source project.
“Staying as close and true to the original open source community as possible allows organizations to get much more out of Kubernetes without having to buy into a vendor specific version,” said McLuckie. “It also makes sure they are experiencing what we as a community intended they experience, while driving down barriers of entry to qualification and testing when running an environment.”
Heptio has introduced Sonobuoy to the Kubernetes Conformance Working Group, an entity chaired by William Denniss and focused on providing a conformance/certification program for Kubernetes platforms. The goal of this effort is to ensure that workloads are portable for end-users across Kubernetes platforms. The objective, said McLuckie, is to “To establish Sonobuoy as a useful tool in our conformance arsenal — already OS under the Apache 2 license. We have been fastidious about making sure it is as broadly available as possible.”
Both Ark and Sonobuoy are fully open source and available on GitHub. For those interested in working directly with Heptio, pricing will vary depending on the type of engagement. “We have support options as well as a team of solution engineers on our customer success team that can work with customers to use Heptio Ark and Sonobuoy for testing and in production,” the company said in a statement.
No matter what implementation route is pursued, one thing is certain: both new Heptio ventures address core challenges that developers face placing containers into production space: How do you make sure it’s healthy, and if it becomes unhealthy, how do you recover?
Ark and Sonobuoy give that insight and sense of confidence and notch down that intimidating barrier to Kubernetes entry.
