Tackling the Complexity of Managing Secrets
Over the years and through hundreds of customer conversations, we’ve heard many common pain points as companies scale and grow in complexity.
One of the most prominent themes is the overwhelming nature of configuration sprawl. Our users frequently share stories about configuration files that are hundreds of lines long, and we know the burden of having to copy/paste both between Pulumi configuration and other sources of truth. With long-lived static secrets manually copied this way, this amplifies maintenance workload, increases the likelihood of configuration drift and can open the door to security and operational risks.
Pulumi IaC was built with configurability at its core and can handle environment-specific differences like these easily, especially when only a few applications and environments are involved. But as organizations grow, and the number of teams, applications and deployment environments increases, the volume of configuration data and time required to manage all of these systems quickly becomes overwhelming.
Introducing Pulumi ESC
Pulumi ESC provides teams with a solution to aggregate secrets and configurations from many sources, manage hierarchical collections of configurations and secrets called (“environments”), and use those configurations and secrets across different infrastructures and application services.
Pulumi ESC works hand-in-hand with Pulumi IaC to simplify configuration management, but also works independently from Pulumi IaC, as a solution for managing environments, secrets and configurations for any application or infrastructure project.
Pulumi ESC: A closer look at our design principles
Let’s delve into the fundamental design principles that shape Pulumi ESC:
- Hierarchical and composable: Environments contain collections of secrets and configurations but can also import one or more other environments. Values can be overridden, interpolated from other values and nested as needed. This promotes flexible composition and reuse and avoids copy/paste.
- Universal secrets integration: Support for dynamic configuration providers allows Pulumi ESC to integrate with secrets stored in any other provider. Organizations often use AWS Secrets Manager, Vault, Azure OIDC and/or 1Password, plus many more sources of truth for their secrets and configurations. Pulumi ESC works with these tools to improve secrets and configuration management.
- Auditable: To view the values of an environment, they must be “opened” and this action is recorded in audit logs, including a complete record of how each value was sourced from within the hierarchy of environments that contributed to it.
- Consume from anywhere: The
escCLI and the Pulumi ESC Rest API enable environments to be accessed from any application, infrastructure provider or automation system. At launch, integrations are available with Pulumi IaC, local environment and .env files, GitHub Actions and more. - Authentication and RBAC: Pulumi ESC brokers access to secrets and configurations that live in other systems, so authentication and granular role-based access control (RBAC) are critical to ensure robust access controls across your organization. Pulumi ESC uses the same Pulumi Cloud identity, RBAC, Teams, SAML/SCIM and scoped access tokens that are used for Pulumi IaC today, extending these all to manage access to environments and stacks.
- Configuration as Code: Environments are defined as YAML documents that can describe how to project and compose secrets and configurations, integrate dynamic configuration providers, and compute new configurations.
- Fully managed with an open source core: Pulumi ESC is offered as a fully managed cloud service in Pulumi Cloud (and Pulumi Cloud Self-hosted in the near future), along with the open source pulumi/esc project where the evaluation engine for environments and the esc CLI is developed.
With these features, Pulumi ESC offers a unique solution for configuration management for modern cloud applications and infrastructure. Pulumi ESC can help dramatically reduce the complexity associated with managing secrets and configurations, offering a centralized solution that streamlines workflows across multiple environments. This not only enhances overall security by mitigating the risks associated with duplicated secrets and frequent copy/paste practices but also significantly improves auditability.
With Pulumi ESC, transparency and accountability are front and center, thanks to detailed audit logs and a hierarchical view of environments. This ensures that organizations can easily trace and understand every configuration’s origin and subsequent modifications.
You can give Pulumi ESC a try via the Pulumi Cloud console, by downloading the new esc CLI, or using the new pulumi env sub-commands in the Pulumi CLI. While in preview, Pulumi ESC is available at no additional cost to all Pulumi Cloud users.
You can learn more about Pulumi ESC and get started today at:
