What the Fork, Amazon?
I’ve been trying to gather my thoughts since the AWS Open Distro for Elasticsearch was announced this week, and explain why I think this is such a devastating move for the open source world as whole.
I’d like to take this opportunity to outline some of the often misunderstood nuances of the open source world — and offer some food for thought for how we can continue enabling and sustaining open source, a movement that has largely changed the world for the better.
So I’ll briefly touch on some of the commentary that arose during this whole fiasco, and provide my perspective, on how I think we can do better.
In an excellent post “Company Value Creation and Capture Fundamentals,” Joseph Jacks discusses the fundamental differences between a commercial open source (COSS) company and a proprietary software company. What I also think needs defining or a better understanding in this complex industry is the fine line between COSS companies, and free and open source (FOSS). I think this is an often misunderstood nuance of the open source industry that needs rethinking, in order to ensure a sustainable and viable ecosystem of excellent open source technology going forward.
Fundamentals of Open Source Software
Recently the Open Source Initiative came out with a post to reaffirm the definition of open source, with this past year being an especially harrowing year for the open source community, following changes in licenses by Redis and MongoDB, raising the fundamental questions of what is defined as open source, once again.
On the other hand, corporate giants such as Microsoft, VMware, F5 and Google doubled down on open source over the course of the year to date — putting into question whether you can “buy yourself” a community. Cloudify founder Nati Shalom discusses this trend in the excellent “The Amazon Effect on Open Source” (written before the acquisitions of Heptio, and Nginx). I hosted a panel on just this topic at OpenStack Boston, and also did a talk on the challenges of open source at DevOpsDays Tel Aviv.
I believe the tenets of Free Open Source Software are quite clear and indisputable, reaffirmed in the OSI post, and by Adam Jacob, former Chief Technology Officer of Chef in this post that I largely disagree with, and I’ll outline why.
The core values of free software are embodied in the four freedoms:
- Freedom 0: The freedom to run the program as you wish, for any purpose.
- Freedom 1: The freedom to study how the program works, and change it so it does your computing as you wish.
- Freedom 2: The freedom to redistribute copies so you can help others.
- Freedom 3: The freedom to distribute copies of your modified versions to others.
I think the technologies made available in this format have done wonders for enabling technological advancement for underserved communities, countries, and people, and this code is powering some of the most critical pieces of the world’s infrastructure. A good read on this topic can be found in the a paper called “Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure.”
Open Source and Monetization
One very profitable model to enable this form of open source to continue being sustainable is the distribution model. This essentially refers to an open source project hosted by one of the well-known open source foundations — Apache Software, OpenStack, Linux Foundation, Eclipse — just to name a few, where commercial companies are built around these projects to provide added feature-sets, production-grade support, and maintain the project in the long-term. Projects such as Kafka, Spark, Hadoop, Git, Linux distributions, have been sustained and maintained in such a manner.
And this has been very profitable model for many companies, and has largely been a win-win format of open source, enabling those that want to adopt vanilla versions and do it themselves, alongside those who prefer more stable and supported versions to have a marketplace of vendors able to support their open source distributions. Such open source companies include, the poster-child for profitable open source Red Hat alongside Canonical/Suse, Github, Hortonworks (and Cloudera formerly), Databricks, Confluent, Heptio and many others. (Which the Amazon Open Distro does not fall under, and I will elaborate on why momentarily).
There is another variation of open source, that we need to start acknowledging, and not lumping all open source into the same category — that is equally beneficial and viable, that has for some reason come under attack and scrutiny, that needs a serious rethink.
This is what falls under the category of commercial open source technology. These are technologies built on open source principles looking to leverage the values of open technology, while being backed by a commercial entity to enable the technology to be sustainable in the long-term. A similar analogy that I can think of that delivers much-needed innovation in such a way is social entrepreneurship — commercial medical or environmental innovations have not received the same level of resentment.
The concept and values are clear, where the ability to provide these technologies or innovations at scale require these companies to be profitable or monetarily backed. Examples of such technologies include: Elastic, Redis Labs, InfluxData, Jenkins, MongoDB, Chef, Mesosphere, ScyllaDB, Cloudify, and many others.
I want to take a deeper dive on these technologies. While FOSS technologies are also maintained on the blood, sweat, and tears of many altruistic maintainers, alongside the commercial distro companies, commercial open source companies are often times kickstarted by a founder with an excellent idea, one they want to offer open source for all of its values and benefits, with the downside of having to constantly fend off abusers. I think Shay Banon, CEO and founder of Elastic, summarizes some of what goes into building such a project and company in the first few lines of his last blog post, On “Open” Distros, Open Source, and Building a Company:
In 2009, I sat down and wrote the first few lines of Elasticsearch and open sourced it. I quit my job and spent two years investing in building the product and fostering the amazing community that has been built around it. In 2012 we formed a company, Elastic, around it.
In order to understand some of the politics around open source, I’ll briefly touch on what is classified as a fork. In 2010, when Oracle bought Sun, they bought with it the then popular CI tool written by Kohsuke Kawaguchi, called Hudson. With a falling out between how the Hudson community was being maintained, Kohsuke forked the Hudson community and created a new CI tool called Jenkins. Being highly admired in the Hudson community — he essentially managed to rally the entire community behind him, and Jenkins is largely the most popular open source CI tool today. The commercial company, that Kawaguchi is now employed at that backs Jenkins is CloudBees.
Another example of this is the MariaDB fork of the MySQL community, which also arose following concern by Oracle’s acquisition of MySQL. These are just a couple of popular examples, there are plenty more.
Why the Amazon Open Distro Is a Fork — And It’s Not OK
Amazon’s claims about concerns surrounding Elastic’s commitment to open source is borderline offensive. As a company that has come under much backlash due to little contribution to open source, and many times exploiting their market advantage to roll out managed services of open source technologies, from popular databases — such as PostgreSQL, Redis and MongoDB, and even their former attempt at an Elasticsearch managed service. Which is what eventually instigated changes to licensing by some of the most popular COSS companies.
With the market share that AWS holds, and a captive audience, it’s quite easy to espouse a hard-earned community and clientele with minimal effort. And this is clearly an abuse of trust. These COSS companies offer their code as open source and free of charge to build a strong community of intelligent people with deep expertise in the product, to enable those who do not have the resources to buy premium features to use the core features of the product freely, and to have the code be flexible, customizable, and open. And many companies, even those of means, often times choose a DIY approach without ever paying the COSS companies a single dime — and this is still considered fair use.
To me, what is considered unfair use, and here is where I object to the very dry definition of FOSS in this context, is when a larger, more profitable corporation abuses this openness for their own benefit, and don’t contribute anything back to the community as a whole. This has been the AWS modus operandi for a very long time.
It’s really easy to offer someone else’s technology for free, I suppose.
While Microsoft and Google are in constant competition over the number of open source committers and maintainers they have in-house, AWS is much less inclined. And even with the number of in-house open source developers they employ and technologies they have rolled out, there has been much doubt regarding these corporations commitment to open source (Microsoft, more so than Google — but this sentiment has been changing).
Had Amazon really been interested in truly changing their ways, as Microsoft has proven time and again, and delivering a truly, valuable and much-needed open source community — they could have started like Microsoft and Google by open sourcing some of its own proprietary technology. Goodness knows that open source cloud has seen its share of difficulty from Solaris, to CloudStack, and even OpenStack. Why not open source parts of YOUR core business? Makes me doubt the real motivations behind your “Open Distro.” It’s really easy to offer someone else’s technology for free, I suppose.
And that’s why I understand Redis, MongoDB, and other companies like InfluxData who found themselves having to change their licenses or close source some premium features to be able to build a sustainable business, and protect themselves from such cynical and hostile corporate behavior.
Preaching open source to a vibrant open source company with deep roots in the OSS values — that has been fully transparent about their needs to monetize and maintain a stellar product, and make dubious claims about its authenticity is simply disingenuous. This is Amazon seeing someone’s shiny toy, and just wanting it for themselves. MINE MINE MINE. This is called a fork.
And this is the dark side of open source. It is the adverse effect we would like the world of open source to be devoid of, as it will have destructive effects on everything that has been built and may be conceptualized and innovated under the auspices of open source.
Since sitting in an excellent talk by Joseph Gordon Levitt at Open Source Summit North America in 2017, I can’t stop thinking about ways we can try and make open source more sustainable — whether through royalties to original writers and maintainers, or some kind of merit system, that will prevent this world from being abused, thankless and meritless.
This is the path that AWS is leading us down, and it will take us back light years in advancement if the world will lock down all of its open innovation. It is our job to prevent this from happening, and find a fair system that enables the innovation that open source technology delivers, Adam Jacob has been vocal about this as has Nati Shalom, Guy Korland and many other open source technology leaders, and I couldn’t agree more.
We need to bring this discussion to the forefront if we want to enable open source in the long term.
