Why the Best Open Source Is Community-Led Open Source
Amazon Web Services (AWS) sponsored this post.
Open source software has been growing in importance for decades, but it hit escape velocity when large web companies started to contribute code in earnest. Suddenly the world was awash in the most innovative infrastructure money didn’t need to buy. Companies like Yahoo! (Apache Hadoop), Facebook (Apache Cassandra), LinkedIn (Apache Kafka) and Google (Kubernetes) released incredibly promising software, inviting a global community to help them to improve upon their creations.
The industry responded. While at times a predominant vendor arose to maintain a particular project (e.g., DataStax for Apache Cassandra), contributions to these and other such projects have tended to come from a variety of corporate and individual developers. Take Kubernetes, for example. Though Google founded the project, more than 35,000 individuals from over 2,000 companies have contributed to a project that now runs to nearly two million lines of code. Perhaps most promising is how not all of these contributors are technology vendors (e.g., Ant Financial, the highest-valued FinTech company in the world and Meituan-Dianping, a large Chinese group buying website, are both among the top-20 largest contributors) — what this means is customers increasingly shape open source software to meet their needs.
If you are concerned about the sustainability of open source, these community-led projects offer hope. While there are, of course, excellent open source projects like MySQL that rely wholly or predominantly on a single vendor (Oracle, in MySQL’s case), open source is at its most innovative and sustainable when maintained by a community. Rather than a single vendor hoping to hoard the benefits of a comparatively small “pie” for itself, community-led open source development fosters abundance, creating a much bigger “pie” for all, including greater revenue potential for individual vendors.
Consider Linux, the open source operating system that powers the world’s fastest supercomputers, most websites we visit and even our phones.
Learning from Linux
Linux has always been a community affair, but it wasn’t necessarily predestined to be such. Linux founder Linus Torvalds originally worked for Transmeta, an early fabless semiconductor company, from 1997 until 2003, when he joined Open Source Development Labs (OSDL), which later became the Linux Foundation. Torvalds’ employment with the Linux Foundation has ensured a neutrality that continues to benefit all Linux vendors and users, including Red Hat, an open source vendor that first grew to prominence by selling support for Linux.
Red Hat, though synonymous with Linux for many, has never dominated its development. According to the latest data for the 5.5 kernel, Red Hat contributes just 6.6% of Linux code (measured by changesets). In Red Hat’s last full fiscal year before being acquired by IBM, that 6.6% contribution translated into $3.4 billion in revenue. Speaking of IBM, if we add in its code contributions, the Red Hat/IBM combined code contributions expand to 9.8% of all Linux kernel contributions. This, in turn, nets IBM billions more in associated revenue from Linux-powered hardware (not to mention consulting services). Importantly, if Red Hat were to magically own all of Linux development, the value of Linux would drop precipitously. Linux matters as much as it does because it’s an abundant community resource.
Nor is Red Hat (IBM) alone in benefiting from this abundance. Oracle, HPE and a range of other enterprise vendors derive their own billions from selling hardware, services, or software around Linux, as do cloud vendors like AWS, Microsoft, Alibaba and Google. Many other companies (e.g., Facebook, Netflix, etc.) build on and benefit from Linux, generating their own billions in customer value. Critically, those billions would likely evaporate if a single company owned Linux; that company would capture all the value and that value would be substantially less. By some estimates, the internet as we know it simply couldn’t exist without a free, community-driven Linux.
The era of a single company owning an operating system and exclusively profiting therefrom is over. This is in part because we’ve discovered, for example, that it’s better and more profitable for all to have a Linux community that creates a large and growing pie shared by many, rather than a comparatively small, zero-sum pie consumed by one.
A ‘Cloudy’ Future for Open Source: Kubernetes, Firecracker and More
Does this talk of pie sound Utopian, like the sort of thing that could only work when open source software emerges from the mind of a talented individual such as Torvalds? That’s not what the folks over in Kubernetes-land think. Nor does it jibe with a bevy of new cloud-oriented open source projects, like Amazon-contributed Firecracker and Lyft-created Envoy, which are foundational to some of the industry’s most innovative services.
Kubernetes didn’t necessarily start this cloud-oriented open source revolution, but it has certainly helped to grow it. Launched in 2014, Kubernetes has given the industry a common rallying point or standard for distributed systems. This only works as a community-oriented project, one in which Red Hat, VMware, AWS and others also pitch in to make Kubernetes bigger and better. As Redmonk analyst Steve O’Grady said, “In a world in which Kubernetes wasn’t open source, it’s a niche product…”.
In a similar manner, Firecracker offers developers a novel way to “build container orchestration platforms or run loads of containers and need to do so with sub-second latency,” Jared Short, Serverless Inc.’s head of developer relations, wrote. Firecracker sits at the heart of AWS Lambda and AWS Fargate and affords others the ability to build similar services. As with Kubernetes, Firecracker is increasingly growing a community endeavor, with developers at Weaveworks and many others expanding its utility.
Nor is this just about technology vendors.
Lyft developed Envoy, an open source edge and service proxy designed for cloud native applications and moved its governance under the Cloud Native Computing Foundation. Envoy has found its way into a number of commercial products, including HashiCorp’s Consul and AWS App Mesh, with a growing community. Apache Hadoop was useful within Yahoo!, but once it was open sourced, it ignited the big data revolution. Apache Cassandra was a nice project at Facebook, but it has become the backbone of incredibly robust data infrastructures at Apple, Netflix and others, even as these companies contribute back to make it even more potent.
In these and other instances, the best open source software tends to be community-oriented, encouraging developers to contribute their particular expertise to make a project much better than any single organization could.
Open Source Abundance, Not Scarcity
Of course, not everyone agrees. Some startups have turned to proprietary licensing to protect otherwise open source code from being used as… open source code. This is understandable but, in my view, short-sighted. There is much more opportunity for all, including these vendors, in a market made bigger and more important by broad participation, both in terms of developing the software and in operationalizing it for customers.
A Google-only Kubernetes is interesting, but not market-changing. Ditto a Linux controlled by Red Hat — such an operating system never would have outpaced Microsoft Windows, UNIX and every other server operating system.
This is the promise of open source: abundance, not scarcity. As an industry, we should be careful not to squander that promise on zero-sum licensing games that leave everyone — customers and vendors — squabbling over an ever-shrinking poverty pie.
CNCF, HashiCorp, Linux Foundation, Red Hat and VMware are sponsors of The New Stack.
Feature image via Pixabay.