We have some seats left for our WarmUp tonight in Seattle. If you are in town, come on by. We’ll talk about the new reality of distributed app development and management. We’ll explore why containers, microservices and the new thinking about data architectures are shifting the discourse from IT to a more community-oriented way of developing and managing projects. And we’ll talk about the role of Internet-scale companies and how they have shifted the way we think about how to develop and manage apps and platforms on distributed systems.
|March 3, 2015 • 6:30-8:30 PM
Experience Music Project • Seattle, Wa.
Over the past few weeks, we’ve been running posts about managing Internet-scale businesses. We featured posts about LinkedIn’s work with Apache Samza, Twitter’s Aurora, SoundCloud’s Prometheus project and Flipboard’s challenges of running a mobile service on the Web.
We’ll keep running these stories and for good reason: companies get that they have to run at scale to be a success in this Internet age. And whether they acknowledge it or not, companies of all sizes are dependent on the inventions of those who were the first to deal with the volume and velocity of data that is now a problem that practically everyone faces.
The New Role Models
Internet-scale companies invent new tools for the problems they encounter. They are the new role models for companies wanting to have their own scaled-out businesses. It’s not IT giants who are thought leaders. It’s more the people from companies like Instagram, Twitter, Google, Facebook, Netflix and a host of others that provide the templates for the rest of us. Those templates come in the form of open source technologies. It’s obvious why — the tools developers could buy do not work for what they want to do, so they invent their own and open source them. Those open source projects feed companies with new ways to think about the new stacks they are building and managing.
This is where it gets interesting. As the Internet-scale companies open source more projects, they become tool providers, too. Their internal projects become the foundation for startups, SaaS providers and traditonal enterprise companies. Simultaneously, a new generation of tool providers are surfacing from these open source projects and creating their own platforms that again feed into the open source community.
The modern developer tool supply chain has its roots with Internet-scale companies. The tools they developed to build and maintain their business have since become a foundation for many an independent tool provider. Now those tools and platforms are becoming fundamental for a different kind of distributed environment, one that connects data centers and different cloud services. These new distributed systems again appear to have their roots in the need for Internet companies to scale. Of course, containers symbolize the new wave, but Google has been using them for years to help manage their own operations. Google Kubernetes, which they released last year, is a container orchestrsation technology. Apache Mesos is the open source orchestration that Twitter uses to run its data centers.
Docker deserves credit for making containers a true catalyst for a new way to think about distributed systems and the new stack; in particular, the changes in application packaging and how that correlates to orchestration services such as Kubernetes, Mesos and others such as Cloudsoft’s Clocker, Tutum and others.
In any case, these thoughts are on my mind as we get ready for tonight’s WarmUp. Hope to see you there.
The New Stack is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.