OpenStack’s Future Sparks Frank Debate at Tokyo Summit
Should OpenStack be a platform dedicated to running cloud-native operations, or should it focus on hosting traditional enterprises applications? Could it be a cloud platform to meet all needs? Or should it specialize, carving its own spot in the cloud marketplace? These are the tough questions being confronted this week by participants at the Tokyo OpenStack summit.
The Summit has a reputation for open-mindedness and restrained skepticism. It’s the venue where developers first questioned the cloud platform’s likelihood to survive what they perceived as an emerging threat from Docker, but also where some of the same people gathered to embrace and extend Docker with the Magnum container management component.
Last May in Vancouver, the candidness continued when termie, a.k.a. Andy Smith — the co-creator of OpenStack’s core Nova compute component, presented a talk entitled “OpenStack is Doomed and It is Your Fault.”
With tongue only partially in cheek, Smith suggested that large companies are effectively assuming control of OpenStack’s evolution from the open source community, moving it in the direction of their more commercial objectives. He called this trend “astroturfing” — essentially a vendor consortium acting under the auspices of a grass-roots movement.
On Monday, at the Tokyo edition of the summit, three well-respected contributors to OpenStack, including one of the founding members of the original Nebula cloud project for NASA, brought forth what had begun as an email exchange between friends. Citing termie’s talk in May, in a mind-spinning display of frankness that would be inconceivable at a vendor conference, they voiced their concerns that the evolutionary path OpenStack is currently on could lead to a dead end, pinched into a niche in-between VMware, Amazon AWS and Docker.
“What their original goal was with Nova,” said Matt Joyce, DevOps lead for Symphony, referring to Smith and co-creator Vishvananda Ishaya, “is now completely different from what Nova has become.”
“Is that differing vision actually hindering OpenStack from being a cloud platform?” asked Kenneth Hui, Director of Technical Marketing at Platform9 Systems.
Are the Customers Wrong?
Hui cited the story of Cinder, the OpenStack component that pools together storage blocks without the need to describe the locations of those blocks to the services that consume them. While the typical storage presented to Nova (OpenStack Compute) is intended to be ephemeral, disappearing when the VM consuming that resource is terminated, Cinder is a way to enable VMs to access more persistent storage that survives beyond the lifecycle of its consumer.
“I would say [Cinder is] growing way beyond what the original vision is,” said Hui, “and has now become more like a traditional enterprise storage system.” He cited the addition of consistency groups, added to OpenStack Juno in 2014 to enable multiple snapshots of a block storage volume, as the kind of feature that enterprises would ask for, because it’s a part of VMware vCenter Site Recovery Manager. Now, volume replication is being added to Cinder because, Hui and his colleagues argue, enterprises perceive it as already having been defined as a necessity, thanks again to VMware.
“I would argue, if you start doing that … we’re building Cinder to such a point where it’s no longer going to be ideal for cloud-native applications.”
Joyce then brought up the subject of Neutron, OpenStack’s networking component. “What Neutron brings to the table, none of that’s necessary in an asynchronous elastic cloud,” he said. “That’s all specific isolation requirements that start coming in with PCI compliance, with isolation necessity, with high availability tolerances — all that stuff is what you’d see in an elastic cloud. It’s very specific to building in these traditional, enterprise-style networks.”
“I’m going to say something that’ll probably piss off some people… just because customers want it, does that mean that should always be what’s implemented?” — Kenneth Hui
Jesse Proudman, the CTO and founder of hosted infrastructure solutions provider Blue Box Group (now part of IBM), disagreed somewhat. Yes, large storage, hardware, and networking vendors are pushing for these enterprise-oriented additions, he said. But it’s not because vendors are trying to add features that will appeal to their customers, so much as they are responding to existing customers demands that these features to be present.
“By far and large, a lot of those requirements are coming from end-user customers who say, ‘I’ve got a PCI-based application, I want to run it on an OpenStack-based cloud, I need network segregation to pass the audits.’ And that’s how the features get added into the implementation,” Proudman said.
Joyce countered with the contention that the additions to Nova and those to Neutron are actually two different evolutionary paths, which are pulling OpenStack into two distinct customer bases with “fundamentally different workflows.”
“I wonder whether we’re getting to the point where we actually need to have, almost, two OpenStacks,” said Kenneth Hui, “one for cloud-native workloads and one for traditional workloads.”
“I don’t necessarily agree that’s the right way to go, but if you look at where things are headed — I’m going to say something that’ll probably piss off some people — one thing is, just because customers want it, does that mean that should always be what’s implemented?” Hui said.
Citing the famous metaphor about giving customers in the 1900s what they want instead of what they need — faster horses, rather than automobiles — Hui suggested OpenStack engineers may need to take the reins, or the steering wheel, when it becomes necessary to steer the platform in a direction that does not yet coincide with customer demand.
Resilience as a Variable
Furthermore, noting that vendors do offer the advantage of extending the reach of the OpenStack ecosystem, he put forth this jaw-dropping supposition: “what if most of the vendors who are involved don’t know how to do cloud-native? And therefore, what they’re building is actually not able to do cloud-native? And by including all of those implementation details in OpenStack, we’ve now created a platform that cannot scale elastically, and never will?”
Hui believes that, among the top three reasons organizations evaluate OpenStack, one of them is the opportunity to replace VMware and its hefty license fees. But in the course of that evaluation, they may (falsely) be perceiving OpenStack as a “free VMware,” or analogous to how Linux was first perceived compared to Unix.
“More likely, what’s going to happen,” he said, “is that there’s going to be this growing group of people in the OpenStack community who’s going to go, ‘you know what? I want it to be free VMware. For now, I don’t care about the cloud-native stuff. I don’t need an AWS in my data center. What I want is a free version of VMware, and I’m going to push all the developers to write code to allow me to fail automatically — do everything VMware can do, but have it be open source.’”
“And that’s going to create problems,” Hui continued, “because that’s not what the original intent of OpenStack is. That’s likely where we’re headed, whether people like it or not.”
“Hot failover was never in the initial design requirements for OpenStack,” Matt Joyce added. “And now it’s a much sought-after feature. That is not ephemeral. By its very nature, it goes against what OpenStack initially was intended to be: an asynchronous, elastic format. But customers want it. This is what customers are asking for. So it’ll be built.”
Citing the success of Netflix’ Chaos Monkey, Joyce argued that what data centers at scale require most is resilience. Customers’ key requirement from day-to-day, their OpenStack wish lists notwithstanding, is the unimpeded ability to transact with customers. Even a temporary inability to fulfill that requirement, he said, could be permanently disastrous to an organization.
“For people to build resilient applications, the idea behind ephemeral was to enforce the idea that everything is ephemeral,” he said. “Nothing is too big to fail in technology.”
But Proudman brought up the issue of whether resilience is a variable, depending upon the critical nature of the application at hand. Bank transactions may need to be completely resilient, whereas a form that takes reservations for a conference room, Proudman suggested, could go down for two hours without triggering the apocalypse.
“There are classes of applications that have evolved over the years,” he said, “and certain classes don’t need to be that resilient.” While many applications should be written with such design patterns in consideration, he continued, “at the end of the day, are we adding too much complexity, or so much complexity by following those patterns, that we make the environments unmanageable and unmaintainable?”
Caught in the Middle
In response to an attendee’s question, Hui pointed out what he perceived as two risks in OpenStack pursuing traditional and cloud-native customer bases simultaneously. First, if a platform makes the design choice to support certain types of existing applications, that decision may render its future support of new applications difficult or impossible.
Second, Hui sees a potential danger in OpenStack paying too much attention to supporting existing applications: that cloud-native developers could say, “‘let’s just skip OpenStack completely. They’re stuck trying to do traditional apps; let’s just go straight to containers.’”
“I think there’s a risk if we divide our attention between those two types of clouds,” he warned. “We may find ourselves caught in the middle, where we don’t do either particularly well, and basically VMware continues to own the traditional space, Kubernetes and Mesos and Docker own the cloud-native space, and OpenStack becomes this very niche player in this very small market that would actually try to use it to do both.”
VMware, Docker and IBM are sponsors of The New Stack.
Cover image from the Nantucket Historical Association Library, via Wikimedia Commons.
