Reaching 1.0, HashiCorp Vault Comes into Its Own
After nearly four years in development, HashiCorp has announced general availability of version 1.0 of its secrets management and data protection tool Vault.
It comes with new features including an open source version of Cloud Auto Unseal, which the community had been clamoring for, according to Armon Dadgar, co-founder and co-CTO, and batch tokens, a new kind of token for high-scale and ephemeral workloads.
The company announced the Vault 1.0 milestone back in October at its HashiConf 2018 in San Francisco. It was just one of the enhanced capabilities across its product suite aimed at reducing management complexity for multicloud environments.
Explaining why it has taken so long to reach 1.0, Dadgar said there are four checkboxes Hashicorp products must meet:
- That the company understand all the major use cases and supports 95 to 99 percent of them.
- The technical architecture has reached a point of maturity.
- And that it has enough time in production, has enough users banging on it, finding the bugs and getting them fixed.
An open source version of Cloud Auto Unseal, which was born in its enterprise offering, has been one of the most requested features among users, Dadgar said.
Unsealing in the past, he said, has been like the scene in the movie “The Hunt for Red October,” where launching a vessel meant two people had to insert keys and turn them simultaneously.
“If I’m a hacker and I get access to your key, I can’t single-handedly get access to the data. It’s hard to do manually,” he said of the process of storing and reassembling Shamir’s keys for all users.
Its solution is to integrate with cloud services such as AWS KMS, Azure Key Vault, and GCP CKMS to automate the unseal process for Vault.
“It comes down to leveraging the cloud providers — who have access to those keys, configuring it so that the cloud provider is the only one that can unseal Vault. You have a single-purpose key, which is to unseal Vault, and that’s it. It ends up being a very narrow surface area and you can use the cloud-native permissioning systems,” he said.
Auto Unseal and Seal-Wrap — data encrypted and stored at rest protected by two different sets of keys — will remain part of the enterprise offering.
The new batch tokens are designed for high-scale workloads, such as Big Data, and ephemeral workloads, such as serverless.
“If I’m going to run a machine learning model or every night I’m going to process a bunch of data, all of a sudden, I might have hundreds or thousands of machines all trying to spin up at the same time and access the credentials. You try to balance demand because my data pipeline is running across many machines. The challenge is a ton of work that shows up all at once on many, many machines,” Dadgar explained.
Meanwhile, serverless environments create really ephemeral-type workloads.
“So that workloads winds up being super bursty. You might go from no workloads to thousands of requests, back to zero.
“Both of these are kind of difficult patterns. You either have a very high-scale pattern or a very bursty pattern. Both of those used to cause Vault some challenge because it was trying to keep track of all the users. The batch token is designed for those two problems. It allows Vault to deal with those two patterns without overwhelming the Vault server.”
He explained it this way:
In a standard mode, Vault is tracking all the information related to the user. If I log into Vault, it sends a cookie showing that, but internally, it’s tracking me, the things I have access to. The problem with all that metadata tracking, if you have a lot of users, that’s a lot of metadata for Vault to handle. If I have a bunch of users, but then they disappear, that metadata is useless.
Instead of Vault tracking that information itself, with the batch token, it encrypts all that metadata and hands it back to the client inside the cookie. The client doesn’t know what Vault has given because it’s encrypted. But when it interacts with Vault, Vault decrypts that metadata and processes the request without having to track all that data on the server side.
These tokens do not write to disk, reducing the performance cost of any operation within Vault, but also are ill-suited for operations requiring persistence or resiliency in the face of the failure, the company warns.
Other notable features in version 1.0 are supported for the OpenAPI standard, which provides a vendor-neutral description format for API calls.
“It lets you automatically generate client libraries for every language. Rather than having to come up with a custom SDK for Javascript, PHP, Nodejs and all the rest, you can auto-generate a library from that Open API specification. It allows us to have a more consistent set of libraries, but also a broader set of communities,” he said.
The other is tighter integration with Alibaba Cloud. The Alibaba Cloud Auth Method is now a supported interface for Auto Auth within Vault Agent.
Reaching 1.0 does not mean Vault is complete, Dadgar said.
“This is the beginning of the Vault journey,” he said. “We see Vault as a security platform. We’ve designed it to be very plug-in-based. As customers have different problems, they can extend Vault by enabling the different plug-ins we provide. If you start with secrets management, how do you want to solve certificate management or SSH brokering? How do we let you quickly and easily extend Vault to provide those other things? “ he said.
In that vein, the company is looking at adjacent problems that plug-ins can solve with the core platform.
Twistlock is a sponsor of The New Stack.
Feature Image: “Key in Flashlight” by zeevveez. Licensed under CC BY-SA 2.0.
