Amazon, Elastic and the Fight for Open Source Freedom in the Enterprise
Amazon Web Services’ recent decision to fork Elasticsearch, a popular analytics engine, and Kibana and offer what it calls its OpenSearch project has raised eyebrows and concerns by some in the community for a number of reasons. Other observers, however, say AWS’ move was necessary to counter Elasticsearch’s commercial model for its use of these open source projects.
AWS is certainly legally able to take the code built from years of devotion and hard work devoted to open source projects, while rebranding the open source tools and platforms and offering paid services to help use and manage the code (depending on the licenses). However, the cloud giant runs the risk of being perceived as betraying its customers and those who helped to create the projects, some observers say.
In the eyes of Elastic CEO Shay Banon, AWS move seems to represent a hostile attack. “Forks that don’t have a driving vision or are born and built on misinformation and negativity tend to fail. Such early traits make the foundation of the community weak, and directly inhibit the pace and quality of innovation — this is not how successful open source products — fork or otherwise — are built,” Banon said in an email response. “It is disappointing that Amazon decided to prioritize their corporate interests over the best experience of their customers, instead of collaborating with us and participating in our thriving community.”
AWS did not respond to queries by The New Stack through a third-party AWS communications representative to comment for this article. In a blog post about the release, AWS’ Carl Meadows, Jules Graybill, Kyle Davis, and Mehul Shah collectively wrote:
Our goal with the OpenSearch project is to make it easy for as many people and organizations as possible to use OpenSearch in their business, their products, and their projects. Whether you are an independent developer, an enterprise IT department, a software vendor, or a managed service provider, the ALv2 license grants you well-understood usage rights for OpenSearch.
AWS’ Intent
Some observers speculated that AWS decided to offer OpenSearch after Elasticsearch began commercializing its open source project. Such a move would not be surprising.
Indeed, Elasticsearch has been “expecting” AWS’ move “for a while now,” Banon said. “When we made our license change earlier this year, we did it after years of behavior from Amazon that was simply not okay,” Banon said. “Our license change preserved free and open access to our products, but limited Amazon’s ability to provide our products as a service without contributing back or collaborating with us. Other successful open source companies have also adopted a similar path in recent years.”
AWS could, in theory, seek to develop commercial paid-for services for OpenSearch in a lopsided way, while neglecting to contribute properly to the open source component. However, such as scenario would represent a “worse-case scenario,” Torsten Volk, an analyst for Enterprise Management Associates (EMA), told The New Stack. Such a ploy would also not live up to its commitments for the open source project and would make “little sense” commercially, Volk said.
“AWS’ move is purely a rational response to Elasticsearch changing their licensing model to one that does not allow Amazon to offer Elasticsearch as a managed service. Elasticsearch is too much of a standard for cloud native analytics for Amazon to just let this go and encourage customers to move to commercial alternatives” to the ELK (Elasticsearch, Logstash, Kibana) stack, Volk told The New Stack. “Maintaining this momentum is important for Amazon to show differentiation in the battle against arch-rivals Google and Azure.”
It is also important to consider how Elastic, the company behind Elasticsearch, has recently drawn $162 million in venture capital “and, like Amazon, cannot be regarded as a non-profit organization,” Volk said.
“The same is true for basically any open source company, as software developers have this nasty habit of not working for free. Therefore, Elastic needed to change the licensing model to make it harder for Amazon to capitalize on their code contributions and to push their own managed service,” Volk said. “Following these harsh realities, neither Elastic nor Amazon had much of a choice and I don’t think Amazon will be more likely than most other software vendors to exploit their new reign over open source Elasticsearch by adding closed-source premium features. There’s little need to do this as Amazon will be looking at the much bigger picture of making it as simple as possible for everyone to onboard and run their cloud native apps and consume as many of Amazon’s approximately 300 services as possible.”
-
The quickly rising developer popularity of AWS Elasticsearch (Source: Enterprise Management Associates (EMA).
However, Banon countered — while noting that the company is “proud of the success we’ve had and in how we’ve achieved it” — that by building a successful business around its products, “we’ve been able to invest more and provide world-class products that are free to use and developed in an open and collaborative model,” Banon said.
“Our goal is to enable and collaborate with the whole ecosystem built around our products, including public cloud providers. Our strong partnerships with Google Cloud, Microsoft Azure, Alibaba, Tencent and smaller providers like CleverCloud, show that this can be done,” Banon said. “In fact, Amazon is an outlier in this regard — choosing to create a fork rather than working together to give their customers the best possible experience.”
The Open Source Spirit
In what can be seen as a similar move, AWS courted controversy in the open source community in 2019 when it began to offer MongoDB as part of its cloud offerings and to offer commercial support services for MongoDB. MongoDB, meanwhile, now offers commercial services based on its open source stack. (MongoDB declined to comment for this article). Earlier this year, Elastic opted to forgo Elasticsearch’s and Kibana’s Apache 2.0 licenses and replace those with an “Elastic License” and a Server Side Public License (SSPL), thus initiating a more commercial business model for Elasticsearch by Elastic.
Since Elasticsearch has already made many of the services proprietary, the extent AWS is remaining true to the spirit of open source with its move last week becomes murkier, while the past can offer some insight about contingencies. In the beginning days of open source, there was much hostility for open source software (OSS) vendors that would just commercialize certain features of a project, Robin Schumacher, Vice President at Netdata, told The New Stack.
“I was at a conference many years ago representing the OSS company I was with and I had one guy who kept literally screaming at me, ‘you should make everything free!” to which I finally responded, ‘when you stop taking a paycheck, so will I.’ He got the point and left,” Schumacher said. “Today, OSS communities are much more accepting of vendors having commercial parts to their offerings because they realize that, without them, the business will most likely fail and the OSS project will suffer afterwards.”
What makes an open source project “idea” and true to the spirit of open source should also likely increasingly come under debate. While noting that forks are “a natural part of” open source, for example, Banon also said he “deeply believe that great examples of successful forks are ones that strive to benefit the community or solve a specific need.”
“The Jenkins fork from Hudson after challenges with Oracle jumps to mind [as an example of a successful fork],” Banon said. “Another one that holds a special place in our history is Grafana.” (Grafana declined to comment for this article).
Several years ago, Sun Microsystems (and later Oracle) “had something similar on their hands when they acquired MySQL many years ago, although Amazon’s move is different,” Schumacher said. “What is more game-changing are the new licenses being put forward by companies like Mongo and Elastic as they try to ward off the sometimes predatory practices of the cloud vendors.”
As it turns out, Oracle ended up being a “very good steward of MySQL and supporter of OSS and so those fears were not realized,” Schumacher said.
“I think Amazon needs time to prove itself in the same way. OSS users are naturally afraid of big vendors who seem to hijack a project and not invest in it, but only exploit it for pure commercial gain,” Schumacher said. “While they have the freedom to do that, vendors seemed to have learned that such a strategy is not best in the long run. Instead, they act in a more prudent manner where they smartly invest in the foundational aspects of their OSS core while commercializing enterprise/advanced functionality that even the staunchest OSS user recognizes is worth paying for.”
