Pulumi Takes Infrastructure as Code Universal with CrossCode
-
Joe Duffy
Joe Duffy has been on a mission to make Pulumi a premier cloud engineering platform and with the latest release of new features taking Pulumi Infrastructure as Code (IaC) universal, he has pulled it off in spades.
Duffy, co-founder and CEO of Pulumi, launched the company with a focus on providing infrastructure teams, developers and security engineers with a software engineering process for delivering modern cloud applications faster and better.
-
Eric Rudder
A former Microsoft DevDiv bigwig himself, Duffy joined forces with Eric Rudder, another Microsoft developer division legend (a former Microsoft CTO and executive VP, now executive chairman of Pulumi), and others to start Pulumi in 2017. They also tapped Luke Hoban, another former Microsoft software engineering wunderkind (and co-creator of TypeScript), to serve as CTO.
CrossCode Is Key
Underlying the move to make Pulumi IaC universal is the Pulumi CrossCode translation technology the company introduced at its PulumiUP virtual conference last week. Pulumi CrossCode is the universal translation layer to Pulumi’s infrastructure as code engine that enables development and operations teams to build infrastructure in most popular programming languages; convert any infrastructure as code format, including Terraform, CloudFormation, Azure Resource Manager, and Kubernetes configuration, to Pulumi; and interoperate with all existing infrastructure managed by other infrastructure as code systems.
Indeed, CrossCode is what enabled Pulumi to provide newly added support for Java and for YAML. Pulumi now supports any Java Virtual Machine (JVM) language (Java, Scala, Clojure), adding to existing support for .NET (C#, F#, PowerShell), Node.js (JavaScript, TypeScript), Go, and Python. Pulumi also announced support for YAML, delivering a simple declarative format for producing or consuming infrastructure as code, enabling simple use-cases and improved toolability.
Pawel Dolega, CTO VirtusLab, said VirtusLab applied its JVM ecosystem experience to Pulumi to help bring support for Java and other JVM languages to the Pulumi open source project and now to the mainline product.
Moreover, with new support for Java, Pulumi extends its platform to millions of Java developers. Java is currently in use by more than 35% of developers, according to the 2021 StackOverflow Developer Survey.
“We’re supporting Java as a new language in the Pulumi ecosystem. And it’s not just Java language, but the entire Java ecosystem,” Duffy told The New Stack. “So that includes any JVM language, including Scala, Clojure, Groovy and Kotlin — we’ve seen a lot of interest in Kotlin. We’re supporting all the tools, all the package managers, like Maven, all the test frameworks… and all of the 80-plus different cloud integrations we’ve got are all available to Java.”
CrossCode is composed of Pulumi’s open source code generation, program generation, schema, and packages software.
Essentially, “CrossCode is basically a language- and cloud-agnostic type system that enables you to project to these cloud providers — of which we have many — anything that can be schematized into all these languages at the same time, with full fidelity support across all of them,” Duffy said. “That’s how when we brought up Java, on day one, we could support the entirety of the Pulumi platform. It’s all thanks to CrossCode. CrossCode is a library, you can depend on it.”
Over time, Pulumi will seed an ecosystem of cross-language tools that depend on this technology.
Secret Sauce
The CrossCode multilanguage technology is really the “secret sauce” to how Pulumi works, Duffy said.
“CrossCode translates the same infrastructure and policy building blocks between different languages. This is the foundation for consistent application stacks between languages and it also provides a ‘bridge’ to YAML,” said Torsten Volk, an analyst at Enterprise Management Associates (EMA). “Having Java guys, Python guys, and JavaScript guys all-access consistently configured, audited, and governed infrastructure objects for AWS, Azure, GCP, Kubernetes, etc., is a big deal in a world where app stacks include dozens of, often cloud-specific, layers. CrossCode provides consistent control between languages to allow developers to write applications and infrastructure that go together optimally without creating an uncontrollable management nightmare.
The Microsoft Connection
-
L to R: Luke Hoban, Eric Rudder, Joe Duffy
Both Duffy and Rudder worked on the Component Object Model (COM) and the Common Language Runtime (CLR) while at Microsoft. COM is Microsoft’s old binary-interface standard for software components that was created to enable inter-process communication object creation in a large range of programming languages. The CLR is the virtual machine component of the .NET Framework. It facilitates the design of components and applications whose objects interact across languages.
“COM and the CLR were/are basically about portability and operation across different environments, so the CrossCode translation is relevant, albeit at a fundamentally different scale given the relative size of Pulumi versus Microsoft,” said Stephen O’Grady, an analyst at RedMonk.
Still, the shared history came in handy.
“Obviously, we share a history with Microsoft. And in my early days, I worked on COM,” Duffy said. “COM was a multilanguage runtime with a lot of similarities to kind of what we built with CrossCode here. Obviously, I worked on the Common Language Runtime and there are similarities there as well. Both Eric and I have worked on these sorts of runtimes many times in our careers. There are a lot of lessons learned that we’re applying to what we’re doing with Pulumi.
Duffy and company started Pulumi by forking the V8/Node.js runtime. Very quickly they decided they wanted to go beyond JavaScript to support other languages and they knew they would eventually need to build a multilanguage runtime but were not certain what shape it would take. Then about two years ago, the company began working in earnest on what would become CrossCode — allowing key customers who needed multi-language support to try it out and help to tweak it.
“I’m so thankful that we had the CLR and COM experience to draw from,” Duffy said.
To that point, the key idea of CrossCode is similar to COM, but the context is a different one,” said Volk.
“The CrossCode layer enables Pulumi’s CrossGuard policy as code platform to monitor and enforce the same compliance packs across Python, Java, JavaScript and the other supported languages,” Volk told The New Stack. “This gives you the ability to make changes to one universal compliance service that will then enforce your specific set of controls consistently across languages. This allows you to declare a set of requirements that must be met for an application to build successfully, in Java, Python, etc. Based on these declarative requirements you can then offer automatic remediation, approval workflows, etc.”
No YAML Zone?
Meanwhile, the new support for YAML, Pulumi enables a simpler, industry-standard markup format for expressing infrastructure as code, that continues to tap into its multilanguage ecosystem, the company said.
“A lot of people think of Pulumi as the ‘No YAML Zone’ or something, but the L is for language, right?” he said. “We’re happy to welcome every language to the platform, but we really understood the importance of YAML.”
Modern Language Support
To build its Blockchain Platform-as-a-Service, SettleMint needed a modern infrastructure as code platform that could meet the demands of multiple clouds, blockchain protocols, and distributed architectures, said Roderik van der Veer, CTO and founder of the company.
“Pulumi’s support for modern languages along with its programmatic API interface are a unique and powerful combination that enabled us to dynamically automate infrastructure provisioning for the SettleMint platform,” he said.
New Packages, Components
Pulumi also added 30 new packages including support for Oracle Cloud, Databricks, and EventStore, adding to support for Amazon Web Services, Microsoft Azure, Google Cloud, Kubernetes, Auth0, CloudFlare, Confluent Cloud, Datadog, DigitalOcean, Docker, GitHub, Kong, MinIO, MongoDB Atlas, PagerDuty, Snowflake, Spot by NetApp, and SumoLogic.
Additionally, new components include out-of-the-box support for container applications, Kubernetes clusters, serverless applications, and more. And a new AWS Cloud Development Kit (CDK) adapter allows any CDK package to be consumed from Pulumi.
“With these new features Pulumi endeavors to meet a wide variety of IaC practitioners where they are, whether they are developers who prefer programming languages such as Java or operators who are more familiar with YAML,” said KellyAnn Fitzpatrick, an analyst at RedMonk. “As infrastructure setups become more varied and complex, and organizations struggle to staff and skill the teams that provision and maintain it, Pulumi is making it easier for their users to leverage existing skillsets and expertise. The addition of support for Java programming languages is especially important to the enterprise given its historic affinity for Java.”
All of these new capabilities integrate with the Pulumi Cloud Engineering Platform, which includes reusable multilanguage components, secrets management, CI/CD integrations, policy as code, and the Pulumi Registry, the company said.
