Linkerd Graduates CNCF with Focus on Simplicity
The Linkerd service mesh, the first service mesh to join the Cloud Native Computing Foundation (CNCF) back in 2017 as the foundation’s fifth project overall, has reached the graduated tier of the foundation.
As a graduated project, Linkerd has met a number of requirements around stability, adoption, maturity, and governance, and joins more than a dozen other graduated projects, such as Helm, Prometheus, Envoy, and Kubernetes.
In a press release regarding Linkerd’s graduation, Justin Turner, director of engineering for supermarket chain H-E-B is quoted as saying that they didn’t “choose a service mesh based on hype,” and that they “weren’t worried about which mesh had the most marketing behind it.” The service mesh being alluded to here is Istio, which, in the most recent CNCF survey from 2020, Linkerd followed closely behind in adoption, alongside Consul.
“Linkerd faced really intense competition for an open source project. It’s basically gone up against projects that have commercial backers with market caps in the trillions,” said Buoyant CEO William Morgan. “And the fact that it has attained graduation, that it has this community of enthusiastic and committed adopters, I think it’s pretty remarkable given that context. It’s hard not to talk about Linkerd without also talking about Istio, although I think the reality is, there’s some pretty fundamental philosophical differences between those projects.”
Linkerd was created by Buoyant in 2016, and Morgan said its first iterations were rather complex before the project found its focus on simplicity. This simplicity, which starts with Linkerd using its own purpose-built proxy rather than relying on a larger, more general-purpose proxy like fellow CNCF graduate Envoy, is a key differentiator for the service mesh, and one of the fundamental philosophical differences Morgan speaks of.
“Naturally, as engineers, what you want to do when you’re building infrastructure is, you want to solve every possible problem with this beautiful platform that can do all things for all people,” Morgan said. “I think when you go down that path, which feels very natural to an engineer, you end up with something that is really unwieldy, and that’s complex, and that is fundamentally unsatisfying. It sounds great, but it’s so hard to operate that you never accomplish your goals.”
Part of the balancing act, said Morgan, is to deliver all the features of the service mesh around reliability, security, and observability, “without getting mired in all the complexity, without having to hire a team of developers or a team of engineers, service mesh experts, just to run your service mesh.”
In the past year, Linkerd has seen a 300% increase in downloads, and part of that acceleration may be attributed to a migration away from Istio due to its complexity. Rather than focusing on moving away from Istio, which he says some users may end up using simply because they see it first, Morgan again focuses on Linkerd’s simplicity as the reason behind its increased adoption.
“In the absence of having these marketing bullhorns, these huge marketing budgets, the way that Linkerd has grown has been by word of mouth,” said Morgan. “It’s been like the way that open source projects used to grow. The way that we’ve been able to accomplish that is by having a really clear vision and a really clear message around simplicity.”
Another key architectural decision made around simplicity was that Linkerd was made to focus on Kubernetes. An earlier version, said Morgan, was made to work with Mesos, Zookeeper, Kubernetes and others, and they instead decided that they had to go with the “lowest common denominator,” which was Kubernetes.
Linkerd’s decision to go with the Rust programming language, rather than Go, C, or C++, was another distinction for the service mesh in its evolution, and one Morgan stands behind.
“It was a scary choice, but we did that because we felt that the future of the service mesh, and in fact the future of all cloud native technology, really has to be built in Rust,” he said. “There’s no reason for us, in 2021, to ever write code in C++ or in C anymore. That was a pretty scary, risky, controversial decision at the time, but it’s paid off because now we have the adoption to kind of show it off.”
While Morgan calls the project’s CNCF graduation “a nice moment for us to reflect and to be grateful for all the people around the world who worked so hard to get Linkerd to this point,” he says that there is a long roadmap ahead, which includes things like server and client-side policies, and “mesh expansion” to allow the Linkerd data plane to operate outside of Kubernetes.
But when your focus is on simplicity, where do you draw the line on additional features? Morgan said that, as a project designer, you have to ask yourself some questions.
“What is the maximum number of those problems that I can solve, and then the rest, we’re just not going to solve? Like, that’s the stopping point,” said Morgan. “There are going to be use cases that Linkerd is just not going to solve, and that’s okay. For those folks, I do actually sometimes tell people to use Istio. There’s a set of things that Istio can do, super complicated situations, where I just don’t want Linkerd to be able to solve that because it would be too complicated.”