OSCON Roundtable: The Great Infrastructure Convergence
In our current state — somewhere along the continuum between brick and mortar data centers and stateless servers — there are gaps aplenty to be filled and a plethora of companies around to fill them. But where should we head next when it comes to infrastructure startups that provide the workflows, scheduling, monitoring and other tools that keep things running smoothly?
In this episode of The New Stack Analysts podcast, The New Stack founder Alex Williams speaks with Jamie Dobson, CEO at ContainerSolutions, Adrian Cockcroft, technology fellow at Battery Ventures, and Kelsey Hightower, staff developer advocate with Google Cloud Platform, about the current direction of infrastructure startups and, more generally, the cycles of technological innovation and how they apply to the stack as it is today and will be tomorrow and in the years to come. The interview was held at the OSCON open source conference, taking place this week in Austin. Listen to the interview on SoundCloud here:
Infrastructure startups find themselves competing in a space where their tasks may be muddled by the continually changing nature of the stack and the applications run on it. Hightower points to a key tension at hand early on in the conversation.
“What role do infrastructure startups play in this migration to the cloud?” asks Hightower. “In the cloud, you see a lot of overlap. How do they survive in that world of infrastructure where most people think the compute is what you pay for? […] Do you pay for the software that allows you to do that? In the cloud, that answer is normally, no. You expect that the APIs to manage that stuff, it’s just part of the cloud platform. Whereas in the infrastructure space, for people selling the tools, that is the value prop – the APIs to manage your compute – and those things are decoupled on-prem, but in the cloud, they’re usually sold together as a complete package.”
Currently, we face a situation where many of the big framework providers may offer many of the required infrastructure tools, but just as many business problems exist without solutions, and startups are rushing in to fill those gaps. And while these various startups may be filling gaps in the meanwhile, the net effect may be a whole lot of wasted effort.
As Cockroft notes, “This is what happens everywhere. Everyone is just stealing the best ideas from everyone else. If somebody says ‘we have this thing’ and you don’t, so they win a deal, then we’ll copy that thing and then we get to not lose against them on that one point. In a competitive environment where everyone’s pitching to the same companies, people are converging on the feature set that matters.”
According to Dobson, it might be after the convergence that we’ll find truly efficient innovation.
“If you have ten companies copying each other it’s totally wasteful,” says Dobson. “There’s actually a lot of evidence that shows that monopolies can be more innovative than competitive systems where there’s multiple companies because they all waste their energies solving the same problem. There’s a lot of that right now.”
The problem for now, however, is that we are currently stuck with old tools for use on new technologies. For example, we use monitoring tools that are intended for monolithic technologies on ethereal microservices, or schedulers that take longer to schedule than the actual lifetime of the process it schedules. When the act of using a tool is actually more complex and time-consuming than that which it runs, schedules, monitors, what have you, then the tool is simply a necessary waste that is begging for replacement.
Nonetheless, we often find ourselves somewhere in the middle ground.
“There’s a fantasy that we can leave the old world behind,” Dobson reminds us. “The history of technology of the past 200 years teaches us that step-wise improvements is how it happens. The world will continue to co-exist.”
Here is a video of the conversation:
Red Hat OpenShift is a sponsor of The New Stack.
Feature Image: Left to right is Kelsey Hightower, Adrian Cockcroft, Jamie Dobson and Alex Williams (sans Fedora).