How Kubernetes Resource Classes Promise to Change the Landscape for New Workloads
This contributed article is part of a series in advance of Kubecon/CloudNativeCon, taking place in Austin, Dec. 6 – 8. Red Hat’s Jeremy Eder will do a deep dive on the Kubernetes Resource Management Working Group at the event.
The Colin Powell rule states that you should make a decision when you have 40 percent to 70 percent of the information necessary to make the decision. With Linux container technology like Kubernetes evolving so quickly, it’s difficult for companies to feel like they have 40 percent of the information they need, let alone 70 percent.
Customers often approach me and others at Red Hat to help them get beyond the 40 percent mark to make a decision about Red Hat OpenShift, which is based on Kubernetes.
For many of these customers, public cloud has become commonplace for workloads. However, translating their on-premise architecture into a proper design/architecture for each cloud is challenging (to say the least) in terms of both time and cost. An architecture that works the same, everywhere, is the promise of Kubernetes and OpenShift, but it’s also one of the heaviest burdens for engineers.
Harnessing Container Potential
I’ve worked with the OpenShift team on v1.0-style containers since early 2012. In 2014, I was approached about working on Docker. It didn’t take long to realize the potential of the technology. In fact, after a few weeks, I’d convinced myself that one day the world’s most critical workloads would run on this next-generation architecture and that we should play our part in making that occur on Red Hat’s product stack.
As part of that stack, the OpenShift container application platform is based on top of Docker containers and the Kubernetes container cluster manager for enterprise app development and deployment. For companies making decisions about OpenShift, it’s especially important to think about how Kubernetes is evolving — and can evolve — to support new workloads.
Kubernetes currently supports diverse and increasingly complex classes of applications. We can onboard and scale out modern, cloud-native web applications based on microservices, batch jobs, and stateful applications with persistent storage requirements. However, there are still opportunities to improve Kubernetes and expand its usefulness and usability. For example, adding support for workloads that require specialized hardware or those that perform measurably better when hardware topology is taken into account would likely increase adoption of Kubernetes in many verticals.
Catalyst for Customer-Focused Change
That’s where the Kubernetes Resource Management Working Group comes in. The group, along with other SIGs (special interest groups), is working to realize customers’ visions while enabling solutions to run well in a fully integrated, thoughtfully planned end-to-end stack.
The Resource Management Working Group traces its roots back to a mailing list post to docker-dev shortly after DockerCon 1 in San Francisco.
After lots of R&D and customer visits, we started throwing around ideas of running high-performance containers. But it was last year that things really started to pick up steam. I pitched the ideas and concepts internally to anyone with ears, including several leadership committees, and was repeatedly green-lighted.
Later, Derek Carr, one of the very first Red Hatters working on upstream Kubernetes, joined the fray. Those conversations provided the seed material for discussions at KubeCon 2016 in Seattle, and out of KubeCon came the idea that we need to formalize ourselves. Derek and Vish Kannan from Google have led the way from there.
We recently had our first face-to-face working group session, and it was very productive. Indeed, it’s hard to pick the biggest technical takeaway, but perhaps the most impactful and wide-ranging is the concept of Resource Classes.
Resource Classes represent a portable way to provide access to hardware resources from applications and are analogous to the proven Kubernetes technology called StorageClasses. Things like HugePages and GPUs fall into this category, but also more exotic hardware like TPUs (TensorFlow Processing Units) and FPGAs. As time goes on and new hardware is developed, the combination of Device Plugins and Resource Classes can be used to onboard capabilities into Kubernetes and provide an elegant and seamless user experience.
Indeed, gone are the days when CPUs were the most efficient place to run every workload. Specialized FPGAs and GPUs are being used more and more to improve performance. Folding specialized hardware into an elastic, software-defined infrastructure makes perfect sense. You don’t need to give up performance for flexibility.
Resource Classes for Real-World Use Cases
This is a fast-moving space, but at the most fundamental level, Resource Classes are meant to provide more granular resource selection metadata to users.
As of Kubernetes 1.9, providing access to cluster-level resources other than CPU and memory requires the use of device discovery, node labels (via Extended Resources), selectors, and even taints and tolerations. As we have worked to enable basic device plugin functionality, users have identified the need for more sophisticated selection criteria. For example, what if I have a workload that runs best on a certain model or manufacturer GPU? What if a node has more than one model GPU in it? What if I need to ensure that a certain workload runs only on an FPGA that is programmed with a certain version of my application?
While it is possible to implement these things using homegrown orchestration scripts, it is logical that Kubernetes could gain some of these capabilities over time to provide a friendly, portable user experience.
Fast-Moving Target
Containers are moving quickly, and Kubernetes, in particular, is moving like a rocket ship. It’s the most popular, community-driven container orchestration project, which can make it hard for users to track progress.
But, at the same time, it’s these users who are driving our work. In fact, it’s only through customer collaboration that we can ensure what’s going on upstream is heading in a truly useful direction. Based on conversations with customers, I provide context to development teams in the community. My team also validates prototypes by running benchmarks and performance tests that reflect customer requirements and goals.
Red Hat is committed to customer advocacy — talking with customers, identifying their needs, serving as a catalyst in communities to implement features on their behalf, and providing information about where the community is headed.
Red Hat is a sponsor of The New Stack.
Feature image via Pixabay.