A Kubernetes Documentary Shares Google’s Open Source Story
It’s one of the great open source success stories — and it’s been condensed into a brief oral history.
Honeypot.io, which touts itself as Europe’s largest tech-focused job platform, also offers a site of programmer-focused videos that it likes to call “Netflix for developers.” Last week, the company released “Kubernetes: the Documentary,” an hour-long documentary (split into two parts) in collaboration with Red Hat, Google, and the Cloud Native Computing Foundation (CNCF).
The video “captures the story directly from the people who lived it,” according to its description on YouTube, featuring interviews with the engineers and managers who built the software and made the crucial decisions that enabled its growth.
On Twitter, the CNCF’s chief technology officer, Chris Aniszczyk, recalled that “It was a bit nerve-wracking having a film crew show up during the pandemic.”
But he added that he was happy to finally be sharing the finished documentary: “We need more of these “tecumentary” style films on open source!”
Google’s Bold Move
There’s even an official trailer, with a shot of early Kubernetes founder Joe Beda remembering “How do change things up — how do we shake the snow globe? … [I]ncrementalism wasn’t going to actually make Google successful with GCE.”
However, Beda’s longer unedited quote in the documentary gives that thought some crucial context. “How do change things up — how do we shake the snow globe in a way that may not be all about Google, but at least gives Google a fighting chance to be able to start grabbing some of these customers, and to start being that balance against the dominance that AWS had at the time.”
It’s a theme that runs through the whole project. On Twitter, Honeypot got a reaction when it shared this quote from the movie: “Google had to make a bold move in the cloud space to be the long-term winner.”
“That quote is from me,” responded Eric Brewer, a vice president of infrastructure at Google, “and I believed it in 2013 and I believe it now. Kubernetes has been a wonderful journey with highs and lows, but in the end, it has changed the game for cloud and computing at large.”
The documentary’s press release points out that, in 2010, Amazon Web Services (AWS) “had a seemingly unassailable lead.” The documentary opens with aerial footage of San Francisco, panning across a server’s network cables while looking back to a time when, as ex-Googler Craig McLuckie recalled it, “Cloud was becoming real, in a very material way.”
But in the next breath, he adds “We were starting to see the ascendance of Amazon as certainly something that was becoming mainstream enterprise technology.”
Today McCluckie, now vice president of research and development at VMware, is credited as one of the founders of Kubernetes — but he’s not the only one who remembers a very daunting landscape in 2010.
Tim Hockin, another early Kubernetes engineer at Google, also says in the film that “AWS was this behemoth that was on everybody’s mind.”
It took them less time for the Google team to finish building Kubernetes’ first open source release for DockerCon than it took to get Google’s approval to open source it.
The documentary then proceeds to chronicle how Google’s forward-thinking engineers “overcome technical challenges, resistance to open source from within, naysayers, and intense competition from other big players in the industry,” according to the video’s description on YouTube.
The story begins with Brian Grant, a Google distinguished engineer, noting that Google had had its own internal infrastructure expertise and hoped to leverage it in the cloud.
But another person who’s pondered that moment in time is Clayton Coleman, Red Hat’s architect for containerized application infrastructure.
In the documentary’s second part, Coleman first says the obvious thing: that Kubernetes wouldn’t have been possible without Docker. But then he adds that Google had also needed to catch up to AWS, while Red Hat’s CoreOS team had believed Linux “needed a good kick to the cloud,” and was already starting to think about what comes next after Linux.
“There’s a lot of what-ifs,” Coleman says. “It looks inevitable in retrospect. It did not feel that way at all.”
The documentary also looks back to the earliest forerunners of Kubernetes, with McLuckie recalling that first moment when Brendan Burns first “stitched together something” which could actually export Google’s in-house expertise to the masses. “It was just scripts and what-not, but I could feel it at that point.
“And for me,” McLuckie says, “that was that moment of recognition that there was something here.”
Deciding on Open Source
In the documentary, Beda acknowledges that “building a community via open source was going to be our best way to essentially establish a de facto standard.” But that wasn’t as easy as it sounds.
In fact, one of the documentary’s highlights is the interviewees’ memories of the skeptical reactions they’d first received from Google’s management — and how close Kubernetes came to not being approved.
Hockin recalls the first response the team got to its prototype after approaching Urs Hölzle, Google’s senior vice president of technical infrastructure, about open sourcing it. “Urs is very analytical, and he said, ‘Why? What do we get out of this? What is the advantage that this brings to Google?’ And so his initial response was: ‘I don’t think we need to do this.'”
“Kubernetes won in part because it had a vast army of contributors behind it. And it just kept marching along, with a huge number of commits per day. And that rate of change kind of trumps everything else.”
—Eric Brewer, vice president of infrastructure, Google
There are many more colorful stories, with McLuckie ultimately recalling that it took them less time to finish building Kubernetes’ first open source release for DockerCon than it took to get Google’s approval to open source it.
But by the end of the documentary, Google’s Brewer is arguing that “Kubernetes won in part because it had a vast army of contributors behind it. And it just kept marching along, with a huge number of commits per day. And that rate of change kind of trumps everything else.”
Creating a Community
The documentary includes footage from the moment Kubernetes was announced at DockerCon 2014 — and open sourced at the same time. “It is yet another orchestration thing that sits on top,” Brewer tells the audience, adding quickly, “That is not a bad thing … there are many of them because it’s an exciting and important area and an area where we need to get agreement.”
“The reason to do an open source release right now, in this space, is because it’s about the ideas.”
Google’s Eric Brewer announces Kubernetes at DockerCon 2014.
They weren’t the only ones. As Coleman remembers it, “Every big startup, I felt like, had a container- orchestration project, and half of them were announced at DockerCon 2014.”
But as Kelsey Hightower puts it, “Kubernetes is what you would build if you had enough time and experience. I doubt anyone had the same level of experience that Google had.” (Hightower, an in-demand conference speaker, is now a principal engineer at Google working on Google Cloud Platform.)
The documentary doesn’t shy away from the feelings stirred up by the announcement of Kubernetes. Beda acknowledges candidly that “as we started looking for partners to get involved, the fact that the project was still dominated by Google, despite us talking about it being open source — it still gave folks a lot of pause.”
Or, as Sarah Novotny says in the film, “Many large companies do not like their employees signing a competitor’s contributor license agreement.”
Novotny is identified in the film as Microsoft Azure’s “open source wonk,” though her past work included leading the Open Source Strategy group for Google Cloud Platform, where she represented Google during the founding of the CNCF — the organization created to address these concerns.
McLuckie tells the filmmakers that “it was probably the most stressful thing I’ve ever done … trying to bootstrap and navigate the multivendor interests in getting something like CNCF off the ground.”
But Novotny recalls that “as we started moving things out to more open governance and as we moved the copyright and intellectual property to the CNCF, larger companies were able to participate. And at that point, it really became an exercise in how to grow the next set of leaders.”
Eagerly awaiting this documentary having lived through this as part of the ecosystem over the last several years!! @NirmataCloud @JimBugwadia
— Ritesh Patel (@riteshdp) October 16, 2021
The documentary also chronicles the growth of the community around Kubernetes. In Kubernetes’ first two years, Grant recalls, “I got 200,000 notifications from GitHub in my inbox.”
Novotny adds that “by the end of 2016, we were having between 700 and a thousand people contributing to each release.”
When Competitors Embraced Kubernetes
Towards the end of the documentary, Beda reflects on Docker’s announcement at DockerCon 2017 that they’d be supporting both Kubernetes and DockerSwarm.
“I think they recognized that this was something that they couldn’t ignore,” Beda says. “And so Brendan and I were there to, you know, say congratulations.”
As he recalls “It was really a cool moment where I finally felt like we stopped pulling against each other and pulling together.”
It wasn’t just Docker. Grant remembers that, in 2017, a lot of competitors embraced Kubernetes — including Mesos — and by the end of that year, even Amazon announced a Kubernetes product.
“For the longest time Amazon had been a holdout on this,” Beda says. “And so to see it come full circle, where it’s like having Amazon supporting it, was one of the largest signals — at least for me — that Kubernetes was here to stay.”
Yet Hightower shares his own perspective, that the container wars “benefited from the fact that there was lots of VC dollars fueling this race — but the truth is, there was no zero-sum game.”
Instead, he adds, the best ideas consolidated into “the latest checkpoint, which is Kubernetes — but the truth is, there’s going to be something that will replace Kubernetes.”