Kubernetes and cloud-based application management are all the rage for enterprise and SMB alike, but Steve Salevan, co-founder of Spine, feels that there is a major gap in the ecosystem. That’s why he created Spi.ne, a hosted service aimed at providing all the DevOps infrastructure needed to move to the cloud to smaller firms that may not have dozens of developers to dedicate to building out such an environment.
Salevan designed Spi.ne using the knowledge he accrued over years working as an SRE at Twitter, Tumblr, Google, and Red Hat, where he worked on test and QA with Ansible creator Michael DeHaan. Last year, just before the election, Salevan left Twitter to build Spi.ne. A year later, and without any outside investment, Spi.ne is in open beta.
Under the hood, Spi.ne is based on Apache Mesos and the Apache Aurora job scheduler. Combined with HAProxy for building out an internal service mesh, and Amazon Web Services for hosting, Spi.ne offers all the internal wiring needed for development teams to gain velocity, without requiring these various items be configured, tied together, and made operational.
Instead, as a hosted offering, Spi.ne simply takes any Docker container and runs it at scale, with full persistent storage inside S3, and automated backup, update, and load balancing functionalities in place. The result is a firm pipeline for developers to grasp onto, ready to go and with no need for a team of dedicated DevOps engineers to build out or maintain it.
Salevan has also built out Spinesible, an open source project based on Ansible which allows teams to quickly build Mesos, Aurora, ZooKeeper, Elasticsearch, and Kafka clusters. It’s Salevan’s way of sharing some of the work he’s done on Spi.ne with teams that may be large enough to implement Kubernetes or Mesos themselves. Spinesible includes many of the Ansible script Spi.ne uses internally.
Salevan compares Spi.ne to Heroku, with the caveat that Spi.ne can run any container, any language, and anything that runs on Linux, rather than focusing on a single language.
While Salevan thinks Kubernetes is a fine project for large teams, he feels that the benefits offered by a completely automated DevOps pipeline transcends the actual benefits of the specific tools used. The benefits come from having that pipeline in place, just as companies like Uber, Twitter and Google have built. The problem is, said Salevan, those companies have large DevOps teams dedicated to building those pipelines, maintaining them, and updating them as they advance. That’s not within the grasp of a firm that may only have 100 employees, and of them, only twenty or so are developers.
“In a lot of cases, developers haven’t had experience running Kubernetes, which requires services like etcd to be stable, or Mesos which requires ZooKeeper; often times developers aren’t used to the failure conditions or nasty operational issues that occur when those systems tend to fail. With Spi.ne, you get the opportunity to not worry about that,” said Salevan.
That’s because this is essentially the same stack Salevan maintained at Twitter. Due to his team’s experience with this stack, it’s highly tuned and ready to scale almost immeasurably, to the same types of traffic numbers Twitter receives.
That’s a tall order considering Spi.ne also provisions its own highly available persistent storage out of S3. Spi.ne also chooses to use Aurora and HAProxy, rather than the more popular, newer Marathon and NGINX pattern. But Salevan said his choices there are based on what he’s seen work at scale. Specifically, he said that the Aurora scheduler has been through some extremely difficult situations at Twitter, so he is sure it can stand up to even the toughest scaling challenges and fail states.
“A lot of people talk about Mesosphere‘s Data Center Operating System (DC/OS) which uses Marathon. We used Aurora instead because it’s battle-tested at extremely high scale. We know the code base have made contributions. Aurora has been dragged through the woods a bit. It has seen some catastrophic failure conditions and has that sort of battle-tested knowledge baked into the code base,” said Salevan.
“Spine boils down the exact feature set you care about and hides the things you don’t care about. Almost all the best practices are there, and it’s providing the ones you need to control, whereas with AWS console, you’re provided with 45 services and it’s easy to configure them wrong,” said Salevan.
Michael DeHaan worked with Salevan at Red Hat building out Funk, the predecessor to Ansible. DeHaan has been watching Spi.ne mature over the past year. While he has not deployed it yet, he is impressed with the platform. “Made rolling updates easy, you just type redeploy and it auto-replaces containers and you don’t have to worry about how that works. It hits the sweet spot between the developer power of PaaS if it was actually achieved with the flexibility of being able to package whatever I want in that Docker container,” added DeHaan.
DeHaan said there is a lot of need for cloud-based services and development support inside small companies. “A lot of the early Ansible customers were startups with five people. If you only have five or 10 people, do you want to even think about running and supporting that level of infrastructure? It’s about giving you a presentable interface. It alleviates the burden of having to set that up and keep it running. You’ve got to have dedicated staff. The ability to get new software out there is massively accelerated with Spi.ne,” said DeHaan.
Salevan sees the data capabilities of Spi.ne as the focus for the next few months. While the system already offers dynamically provisioned persistent storage, Salevan is targeting MySQL as the next platform to bring into Spi.ne.
“The primary focus of our development going forward will be improving our persistence options and building database support which will automatically handle things like slaving and master promotion,” said Salevan. The company is working on a Mesos scheduler, called Mysos, that can create self-sustaining MySQL clusters. “We’re moving into the database field and making databases and large data storage something people can handle,” he said.
Spi.ne is in open beta now, and users can get a free $50 credit to test things out with.
Mesosphere is a sponsor of The New Stack.