HashiCorp Revamps Vagrant and Retools for Microservices
In a frank and revealing keynote address Monday to his young company’s first developers’ conference in Portland, Oregon, HashiCorp CEO Mitchell Hashimoto declared the bedrock product upon which his company was founded outdated, outmoded and too hard to use. Admitting his Vagrant deployment automation tool puts too much burden on users with respect to how software deployments should be repeated across multiple projects, and admitting that customers may have helped him reach this revelation, Hashimoto boldly announced Vagrant’s replacement, officially released today.
Called “the successor to Vagrant,” Otto will subsume Vagrant’s basic automation engine, but will build onto it a new platform that determines the best configuration for a desired application deployment state, based on the status, architecture and capacity of the network it’s being deployed on each and every time. This way, Hashimoto told his first attendees, configuration files won’t quickly become archival records of how applications were best deployed in some distant past.
HashiCorp CEO Mitchell Hashimoto addressing HashiConf 2015
“This is the beauty behind Otto; it’s a key design architectural difference over Vagrant,” the CEO told his audience. “It’s the idea of codification versus fossilization. If you take a Vagrant file that was written five years ago, and you ran vagrant up today [its key deployment command], it would still work, but what Vagrant would do is bring up exactly how you configured that environment five years ago … But for application deployment, it’s not always actually what you want.
“What Otto does instead is codification,” he continued. “What Vagrant does is called fossilization. It makes it a fossil; it imprints it, it snapshots it, and it doesn’t change for all of history.”
Not as Think as You Dead It Was
The notion that development technology is so old as to measure its lifespan in years rather than weeks should be declared “dead” already, was commemorated last June by CircleCI founder Paul Biggar’s blog post entitled, “It’s the Future.” In a hypothetical dialog, the writer consults an expert who’s a serial conference attendee about how to simply deploy an app. The expert replies, use Docker, it’s like LXC, except LXC is like chroot, which is something like Docker.
“Oh, so like Vagrant,” the author says. “No, Vagrant is dead,” replies the expert. “Everything is going to be containerized now, it’s the future.”
If Vagrant’s successor to the throne has already been effectively crowned by Vagrant’s own creator, then is Vagrant really dead? Not really, states Hashimoto in a note to The New Stack just before taking the stage Monday.
“Vagrant is a mature, battle-hardened piece of technology,” writes Hashimoto. “It has been in use by millions of users for many years. We don’t want to reinvent the wheel, so we’ve taken the best parts of Vagrant and used them within Otto to manage development environments automatically for the user.
“Otto builds on top of Vagrant to make operations such as SSH take milliseconds, automatically assign addresses, and more,” the CEO continues. “In addition to all the development environment features, Otto does a lot more to enable deployment. This is what we mean by Otto being built on Vagrant: It uses the maturity of the tool and the wisdom it has gained in over five years underneath. But we’ve added a lot on top of it. Users of Otto don’t need to know how to use Vagrant, Otto does this for you.”
Otto can make use of an appfile that declares the desired configuration state of the application, but does not have to. Even in the absence of an appfile, Otto determines the best possible configuration for the application being deployed, given the constraints of the system it’s running on at the time. But that determination is effectively fed into Vagrant, which continues to work behind the scenes.
“Codification is the idea that we have something high-level enough where the knowledge is centralized in Otto,” explained Hashimoto during his inaugural keynote, “instead of the configuration file. We have this centralization of knowledge of everybody, of the community. As Otto grows, Otto will learn more. So if you take an appfile today and run Otto five years from now, it won’t bring it up the same way. It won’t deploy it the same way. But it’s going to deploy it in a way that’s correct for five years from now.”
I am Nomad. I am Perfect.
Otto’s ability to interpret the state of its environment in real-time will be enhanced by another new component in HashiCorp’s line, released today, though this time not succeeding anything. Nomad is the company’s new real-time scheduler, whose aim is to locate the best infrastructure for supporting any given workload. As HashiCorp CTO and co-founder Armon Dadgar told his audience Monday morning, Nomad will pool together multiple resources from tens or thousands of machines, into a more manageable, virtual cluster. (HashiCorp is certainly not inventing the wheel here, although one could argue the advent of Nomad was vitally necessary for HashiCorp to remain competitive.)
HashiCorp CTO Armon Dadgar addressing HashiConf 2015
“As our infrastructures are becoming larger, and as our applications have gotten more diverse, it’s gotten more and more challenging to deploy things to this,” explained Dadgar. “Using a scheduler provides us with something of a checkpoint — an ability for us to say, ‘Is there a way for us to wrap this complexity, wrap this diversity, and build a layer that’s homogenous so that we can keep building higher-level tooling, and move up the abstractional chain?”
As Hashimoto explains to The New Stack, Nomad’s perception of workloads is intentionally agnostic. “Nomad has a unique feature we call task drivers,” he tells us. “Other schedulers in the space have focused on primarily scheduling Docker containers.
“Task drivers give Nomad a pluggable model for supporting other execution types,” he continues. “This allows Nomad to run things that aren’t in a container or don’t benefit from being in a container. For example, Nomad can schedule static binaries directly. Static binaries are fully self-contained applications with no dependencies other than the operating system. Nomad runs this and automatically puts it in a container for you. Another example that is unique to Nomad is that it can schedule virtual machines as well. For applications with high-security requirements (such as Vault), this lets Nomad execute those tasks in a virtual machine. And these virtual machines can run side by side with Docker containers and other execution types. For enterprises, this means that if they have containers, we can deploy them. But if they don’t, we have a lot of other options for them as well, easing their adoption.”
Beyond Cut-and-Paste
In perhaps the most revealing moment of one of the more refreshingly honest keynotes we’ve seen this year, Hashimoto showed his audience the results of a Google search for, “how to deploy a Rails application.” The #2 result that Google retrieved was an article that appears to fit the bill. That article, published by DigitalOcean (one of HashiCorp’s platform partners) leads the reader by the hand in how to cut-and-paste the necessary instructions into the command line, one-by-one.
“This wasn’t a best practice fifteen years ago!” declared the CEO. “So why is it still a top hit today?”
Surely the cut-and-paste process is simple enough, in that it doesn’t require a college degree or a software certification. This simple process does not account for how developers would implement their own public and private subnets first. Of course, Hashimoto said, HashiCorp already produces seven tools that help the developer or DevOps professional to cope with these specific needs.
But the realization of how steep a learning curve those seven tools impose, he stated, ends up being why developers resort to using Google searches in the first place, and why they end up cutting-and-pasting. If a process can be cut-and-pasted, for countless reasons, it would be better if it were automated.
What’s more, Hashimoto realized, new microservices architectures introduce tremendous dependencies between components and one another, that may very well be impossible to keep track of using static configuration files. For Otto, the first order of dependencies may be stated explicitly within an appfile, but the components upon which those dependencies are themselves dependent is either stated by the files for those second-order dependencies or else determined dynamically.
Although HashiCorp’s faithful contributors are members of the open source community, Hashimoto tells us, both Otto and Nomad were developed in-house. Their code is being released to the community for the first time today.
“We like to release complete solutions so that our community can begin using it right away,” the CEO writes. “It also has a fun effect for ‘fans’ of HashiCorp: They know we’re working on something, but don’t get to see it until the big reveal.”
Docker is a sponsor of The New Stack.
Feature image: “Sapphires In Hot Magma – 2014” by Jurassic Blueberries is licensed under CC BY 2.0.
