Idem: Could VMware’s Salt Project Solve the Multicloud Mess?
DevOps teams know how incredibly complex managing multicloud and numerous APIs is. The process remains one of the major resource-draining challenges organizations face as they scale. Additionally, despite very helpful processes, such as GitOps, and numerous other solutions on offer, each CI/CD production pipeline typically requires massive amounts of configuration and manual management for every environment in which an application is deployed.
A number of alternate alternatives exist that provide interfaces on top of all these different APIs and cloud environments purporting to simplify the complexity. However, they do not solve the fundamental problem: how to automate the largely manual process of parsing through often incredible amounts of code bases associated with each deployment.
This calls for a fundamental shift in how the configuration data is presented and made available for each cloud environment and API so that the data is accessible and processed with actionable results.
During the past couple of years, SaltStack (which VMware acquired in 2020) and VMware engineers headed by SaltStack founder and CTO, Tom Hatch, set out to find a solution, resulting in the open source Idem Project. The resulting open source tool was created to reduce the enormous complexities associated with orchestrating massive amounts of codebases for each cloud deployment and API into a format consisting of data that the project creators say a human mind can readily understand and manage.
The Idem project could potentially solve a “foundational” problem associated with developing microservices applications for multiple cloud environments, Torsten Volk, an analyst for Enterprise Management Associates (EMA) told The New Stack.
“Doing away with all of this complexity without any doubt is the foundation for a universal cloud automation platform,” Volk said. “Whether or not VMware has really pulled this off to a degree that covers the core IaaS use cases might be one of the most interesting things to watch in DevOps in 2022.”
The main issue is how AWS, Azure, Google Cloud, CloudFlare “and friends all offer different APIs to set up, configure and manage the same core services for storage, network, compute, database, messaging and so on,” Volk said. “This makes it tricky and labor intensive to define truly identical application environments. Without identical environments you cannot know upfront whether the same infrastructure code requirements lead to the same results in terms of performance, availability, reliability, security, compliance and even runtime components,” Volk said. “Idem introduces the concept of ‘Idempotence’ to describe the platform’s ability to define and enforce consistent results when executing application code on different clouds. You could describe this ambition as the ‘holy grail’ that provides access to the true promise of multi-cloud.”
Infrastructure as Data
In this way, Idem “reduces” cloud configuration. Instead of Infrastructure as Code, Idem delivers Infrastructure as Data, according to Idem documentation.
Idem offers a “high level of automation that is custom-built to simplify the complex orchestration of multicloud and multiple APIs. We had to look fundamentally at a number of the problems that we’re dealing with and where the real bottlenecks are, which is back to the human because automation was lacking,” Hatch said. “So, let’s just say we’re the people behind the automation that Idem offers.”
VMware acquired SaltStack largely to extend its multicloud and infrastructure automation. For VMware, this automation, with open source Salt and other open source tools under the SaltStack umbrella, was applicable to cloud native deployments for VMware’s vRealize cloud automation software, for example. VMware’s widening cloud reach now also includes, of course, VMware HCX for migration of vSphere workloads.
“One of the goals of acquiring SaltStack was to figure out how to use the same engine under multiple products — and so this is one of the things that’s been exciting is that Idem is already getting used by multiple products inside of VMware and people are already saying ‘everything is a lot easier if you’re not trying to build your own cloud connectors for every API’,” Hatch said.
🚨Idem Project is officially here! 🚨
Instead of Infrastructure as Code, Idem delivers Infrastructure as Data. Browse its new website and start freeing yourself from writing and maintaining a mountain of code!
— Salt Project (@Salt_Project_OS) March 1, 2022
The POP Connection
Idem draws many of its capabilities from SaltStack’s open source Plugin Oriented Programming (POP), introduced at SaltConf2019 in November, POP, among other things, allows developers to work on code in decoupled codebases or repositories. Changes to an application are then automated to extend across several data centers and often multicloud environments in the form of plugins. The standardized plugin configuration is thus applicable to all of the environments, thus eliminating much of the separate operations work in ensuring the new code works across the entire application ecosystem on which the application is deployed.
Idem follows POP’s plugin-oriented programming. “POP is a lot of where we get the power from for Idem: is the fact that everything’s plugged in, everything’s pluggable. We’re using microservices software design patterns that we get from POP for Idem, which makes it really easy for us to be able to create lots of different cloud supports and repos and have it dynamically merged together, but still be really fast and really efficient,” Hatch said. “It makes it really easy for us to get lots of distributed teams on board and make sure that all of these individual components can be written in a microservices-style way, while still being able to deploy the software in an easy and consumable way.”
Deeper Analytics
Idem’s engineering team is working to add more deeper analytics it offers each cloud environment or API to which Idem is deployed. “We are working so that Idem not only inspect and describe a cloud but can then do really deep-data analytics on exactly what’s happening, where your configurations are, where your divergent resources are, and dynamically turn that into really manageable code,” Hatch said. “Idem will be able to take an entire cloud and construct all the knobs and levers that you need in a highly simplified way to just say, ‘push and move’ and your cloud moves with you, which, to say the least, lowers that code overhead significantly.”
