Kubernetes: the Magic is in the Complexity
Any discussion about Kubernetes eventually lands on a common complaint: It’s complex. This is why the Kubernetes learning curve is so steep, and why the developer experience is often considered lacking. Of the many companies and technologies that have sprung up around Kubernetes, a substantial number offer a reduction in complexity as one of their core benefits.
So why is Kubernetes so complex? How legitimate is this complaint? Let’s dive in, because It’s Complex.
Is Complexity a Problem?
“It’s an extremely powerful platform,” said Joe Pelletier, vice president of strategy at Kubernetes services provider Fairwinds, speaking of K8s . “It’s the best platform if you want to run really great operations, but we already know running great ops is still a complicated thing.”
Whether or not complexity is a problem often boils down to how sophisticated the users are. This is more than a question of individual skill — not all engineering teams have the same skill level or are equally as comfortable trying out new projects. In fact, many organizations end up finding that the procedural and organizational changes required to adopt Kubernetes were more challenging than teaching individuals the new technology.
For very sophisticated teams, both technically and in their ability to adopt to new ways of working, the complexity in Kubernetes isn’t likely a problem. As Kubernetes becomes more mainstream, complexity management becomes a serious concern.
Of course, just figuring out how to manage complexity — or wading through the many technologies used to secure Kubernetes, manage resource usage, connect to storage, connect to legacy environments, bridge public and private clouds — adds up to a major endeavor. “Part of the complexity is the sheer number of options available,” Pelletier said.
To a certain extent, it might even be unfair to call Kubernetes ‘complex.’ Kubernetes, after all, is designed to handle apps built with microservices. “These are fairly complex apps to begin with,” said Kaylan Ramanathan, vice president of product marketing at cloud monitoring company Sumo Logic. “It’s not a monolith anymore. It’s lots of services, talking to other services.” To make matters even more complicated, these applications are constantly changing.
So perhaps, given everything that Kubernetes does, the ecosystem that it exists in and the type of applications it was designed to support, it would be unbelievable if Kubernetes were simple. Kubernetes exists to help manage the complexity of containerized microservice architectures.
Yet having such a complex orchestration platform can lead to concrete problems. “The complexity impacts the speed of delivery as well as the speed of recovery when issues arise,” said Bruno Andrade, founder of Shipa, an application management framework on top of Kubernetes.
So what can be done?
Whose Problem is This?
“The way I see Kubernetes, is that it is meant to orchestrate the infrastructure,” explained Andrade. “Kubernetes is great middleware, but for you to use Kubernetes you have to know what a deployment set is, how persistent volumes and persistent volume claims work. I think the problem is we’re dragging developers down to this level.”
Or, as Ramanathan said, “it’s an alphabet soup of things you have to understand to run Kubernetes well.” And that’s just at the application layer.
It’s actually possible that part of Kubernetes’ bad rap for being complicated is due to the fact that developers, who aren’t used to handling infrastructure, aren’t generally able to get a seamless experience in which Kubernetes is totally hidden. It seems complex, because there’s a bunch of stuff they don’t really understand. Think of it like learning a new language: At the beginning it seems extremely complicated, but by the time you’re an expert everything makes sense effortlessly.
“I think part of what we’re seeing in the Kubernetes ecosystem is there [are] a lot of different constituents,” Pelletier said. At the very least, there’s usually a platform or DevOps team primarily tasked with making Kubernetes work. Their ‘clients,’ so to speak, are the developers, who many people think end up being forced to interact with Kubernetes more than they should be. Kubernetes isn’t necessarily complex to the platform team — infrastructure is their specialty. But to developers… yikes.
How Do You Fix Complexity?
First of all, few seem to argue that core Kubernetes itself should be made less complex. Instead, there are different ways to manage the complexity that don’t involve changing the ‘engine,’ just putting in a better steering wheel.
“The practice seems to be that it’s something that is installed on top of Kubernetes,” explained Robert Brennan, director of open source software at Fairwinds. “So that if you do end up growing to a point where you need more control, you can always drop down into core Kubernetes and get your hands dirty.”
The public cloud provider versions of Kubernetes reduce complexity — as do many other commercial projects — through managed Kubernetes services, purpose-built dashboards and a suite of automation tools. Generally these options are much more opinionated than vanilla Kubernetes, but they help streamline things for the platform team as well as the developers. This is often expressed as putting in guardrails to make sure developers don’t mess things up.
At the organizational and procedural level, the key to reducing complexity is through automation: Kubernetes feels much more complex when someone has to turn all the knobs manually, but once the right automation tools are in place, configurations or policies only have to be set once.
Bug or Feature?
It would be hard to argue that complexity is not a core feature of Kubernetes as it was designed. It was intentionally created to be highly configurable, adaptable and extensible. Almost by definition that means that it is very complex. “Nobody has a use case that can’t be solved by Kubernetes,” explained Brennan.
At the same time, it’s equally clear that this complexity is a huge barrier to entry, especially for large organizations that are already dealing with complex application architectures and might not be the most sophisticated, cloud-forward engineering team.
As Kubernetes moves from being a technology used by the organizations at the very forefront of the cloud revolution to being a mainstream enterprise technology, the complexity issue will likely have to be addressed more completely. There’s plenty of evidence that the community is heading in that direction, and that in the future there will be options for both the ‘give me all the knobs’ crowd and the ‘make it easy’ crowd. Users, however, just have to understand that they can probably have either infinitely flexible or simple, but not both at the same time.
The Cloud Native Computing Foundation, which oversees the Kubernetes project, is a sponsor of The New Stack.
Feature image by Artem Maltsev on Unsplash.
