Mesosphere Raises $36M From Khosla Ventures to Build Out Data Center Operating System
Mesosphere will accelerate development of its data center operating system with $36 million in funding it received in a round led by Khosla Ventures, one of the Silicon Valley’s most respected venture capital fims. In June, Mesosphere raised $10 million in a round led by Andreessen Horowitz.
The company has gained considerable attention in the past six months. That is in part due to the founders pedigree as the architects behind Internet scale operations such as Twitter and Airbnb. But it’s also an outcome of the new interest in containers that has come with Docker’s rising popularity.
Mesosphere emerged from Mesos, which Data Center Knowledge described in September as a distributed systems kernel born out of UC Berkeley’s AMPLab about five years ago. Benjamin Hindman was a PhD student at Berkeley at the time, who went on to work at Twitter for four years before joining Mesosphere this Fall. He is one of the original creators of the Apache Mesos project.
Mesosphere’s data center operating system (DCOS) is designed to treat the data center as one giant computer. The Mesosphere difference comes with the philosophy about how they treat the data center. It’s not as much the machines that it supports. More so it is the applications that are running inside the data center. This turns the enterprise model on its head. For years, an IT executive would build out the hardware and then run the software across it, treating all those servers and big boxes as pets. In today’s world, it’s the applications that come first. The hardware gets abstracted, treated more like cattle than anything else. This comparison is used often these days to explain how modern data centers must be treated in today’s application-centric and data intensive world. Apps com first.
The data center is a virtual corral. Resources are pooled and apps are launched much in the same way as operating systems work on computers.
Mesos has the capability to scale the number and size of apps as well as the compute, storage and other resources that are needed for different types of workloads. It’s core is in the kernel, which performs the main functions such as allocating resources to apps.
Modern operating systems provide services to users and applications via a variety of components. These components work together to offer a complete system. In a conventional operating system, the components are separated to improve reliability, security, and maintainability. In DCOS, components are distributed and redundant to provide high availability. DCOS allocates resources to applications, rather than manually distributing load. This allows DCOS to achieve high datacenter utilization and provide stable performance for applications.
The user interface is a newer, noteworthy aspect of Mesosphere’s DCOS.
Mesosphere DCOS has a modern design that has been come to be expected from developer tools and management services for distributed infrastructure environments. The attention to design is evident in its monitoring capability, which offers a high-level view of the health and performance of the data center, recognizes capacity issues, offers recommendations for application tuning and scaling as well as the ability to detects system failures.
The other major capabilities of the Mesos DCOS:
- Init System, also known as Marathon, which determines what apps to launch. It scales networking, messaging and other functions based on demand. Mesos serves as a framework to launch other apps and frameworks. Its API starts and stops other apps.
- Cron, also known as Chronos, serves as the scheduler for the DCOS.
- A service discovery capability keeps track of databases, web servers and other back-end components as they move from node to node.
- The Mesosphere DCOS uses Hadoop as its distributed file system. Mesosphere execs say Hadoop allows the user to run apps that need greater capacity, performance and reliability. Those are all major requirements for the enterprise customer.
- Google Kubernetes is an alternative to Marathon, which customers can run in Mesos. According to Mesosphere, Marathon and Kubernetes can run simultaneously within a cluster if there are competing needs or user demands.
- In Mesos, applications are known as frameworks. In Mesos frameworks entered into a registry can be installed by a single command. This means that a customer may run its traditional apps and frameworks for Spark, Cassandra, Kafka and Yarn. All are well-known open source services that are designed to run on distributed infrastructures.
- DCOS supports Red Hat, Debian, Ubuntu, and CoreOS. According to the company, the DCOS can be used with virtual or bare metal nodes in a private datacenter or with cloud instances on the AWS (Amazon Web Services), GCE (Google Compute Engine), and DigitalOcean cloud services.
Mesos has the support of Google. It received the blessings of Docker last week at DockerCon. Mesosphere is now a first-class citizen in the Docker ecosystem. And now the company has the financial blessing, too, from Khosla, which is known for working with its portfolio companies to develop sound business practices.
There will be hurdles. Internet scale companies like Twitter have a need for what Mesosphere offers but those companies are few and far between. Plus, there’s lots of overlap with similar technologies such as with Kubernetes and services such as Clocker from Cloudsoft and to some extent Cloud Foundry’s Diego framework.
As we wrote earlier this Fall: “The orchestration problem that Diego resolves makes it comparable to platforms such as Google Kubernetes and Mesos. Kubernetes and Diego both run on etcd, the configuration technology that is also what CoreOS uses to manage its OS for server deployments.”:
Mesos is simply a communication infrastructure and protocol for enabling scheduling across distributed resources. Diego does that (admittedly with more specificity) and a lot more: containerization, log aggregation, routing, health-management, etc. Frameworks built on top of Mesos such as Marathon are more directly comparable to Diego.
There are also several new entrant to the market such as Tutum. And end users are also developing their own orchestration technologies to manage their Docker deployments.
But Docker has helped loosen the enterprise mindset by focusing its work on developers and the applications they build. The “cloud” is not really ever discussed by Docker. Instead, Docker executives will discuss how containers are used to accelerate app development. As a result, customers are now looking more at how to abstract their hardware so they can develop apps faster on infrastructure that is treated like cattle, more than pets.
Pivotal, Cloudsoft and New Relic are sponsors of The New Stack.
