A Shortcut to Building an Enterprise-Grade Platform
Designing an effective internal developer platform (IDP) is a time-intensive process. It not only involves determining the essential components needed, but also choosing the most suitable technology amid a constantly expanding and intricate landscape of tools. To help fast-track this process, Humanitec has open sourced the reference architecture implementation codes for Amazon Web Services (AWS) and Google Cloud Platform (GCP)-based platforms.
This latest development enables you to speed up the entire design process and easily build a minimum viable product (MVP) within less than one hour, rather than months. For more support, we launched this learning path on how to master your IDP and set up a reference architecture using Backstage, the Humanitec Platform Orchestrator, Score and your chosen cloud provider. The learning path walks you through the everyday tasks of developers and platform engineers, running applications and modifying the setup in hands-on sessions.
Before we dive into the what and the how, let’s take a look at why there’s a need to build an enterprise-grade IDP.
Slash Time to Market with an Internal Developer Platform
Internal developer platforms have the potential to significantly simplify the software delivery process when implemented effectively. As highlighted by our CEO, Kaspar von Grünberg, “An internal developer platform (IDP) is the sum of all the tech and tools that a platform engineering team binds together to pave golden paths for developers. IDPs lower cognitive load across the engineering organization and enable developer self-service, without abstracting away context from developers or making the underlying tech inaccessible.”
Organizations that use an IDP can drive standardization across application and infrastructure configurations. This empowers developers to self-serve throughout the entire application life cycle, eliminating the need to wait for Ops support, which in turn drives productivity. Reducing Ops bottlenecks contributes to a more efficient workflow. Moreover, IDPs alleviate the cognitive burden on developers by allowing them to choose abstractions while keeping the underlying technology accessible.
From a business perspective, these benefits translate into tangible advantages. When developers can focus more on coding, organizations can slash time to market (TTM) by 30%, achieve four times higher deployment frequency and accelerate lead time by 30%. These improvements are pivotal in fostering revenue growth and equipping enterprises with the agility to respond swiftly to competitors, evolving customer demands and market dynamics.
Why the Need for an IDP Reference Architecture?
When it comes to designing and building an IDP, every platform looks different. And until recently there was no standard, proven, scalable or repeatable pattern for platform teams to follow. How your IDP will eventually look depends on the technologies you already have in place, which ones you want to get rid of and keep, and the golden paths you want to design. The size of your organization, preferred developer workflows and external factors like regulations will also shape the outcome of your IDP. So it’s only natural that different companies take very different approaches to building their IDPs.
At PlatformCon 2023, McKinsey’s Stephan Schneider and Mike Gatto unveiled their latest reference architecture for IDPs on Amazon Web Services (AWS), combining insights and commonalities gleaned from numerous real-world platform setups. Their presentation, titled “Platform as Code: Simplifying Developer Platform Design with Reference Architectures,” inspired us to develop our own IDP reference architectures tailored for AWS, Azure and GCP-based setups. These resources aim to help organizations swiftly design, construct and deploy enterprise-grade IDPs.
Key Platform Architectural Components
There are five main planes set out in the reference architecture that make up different areas of the platform. While the reference architecture’s aim is to give you a starting point, your existing estate should also be considered in each organizational context to incorporate the components already in place:
Developer Control Plane
This plane is the primary configuration layer and interaction point for the platform users. It harbors the following components:
- A version-control system. GitHub is a prominent example, but this can be any system that contains two types of repositories:
- Application source code
- Platform source code, such as using Terraform
- Workload specifications. The reference architecture uses Score.
- A portal for developers to interact with. It can be the Humanitec Portal, but you might also use Backstage or any other portal on the market.
Integration and Delivery Plane
This plane is about building and storing the image, creating app and infrastructure configs from the abstractions provided by the developers and deploying the final state. It’s where the domains of developers and platform engineers meet.
This plane usually contains four different tools:
- A CI pipeline. It can be GitHub Actions or any CI tooling on the market.
- The image registry holding your container images. Again, this can be any registry on the market.
- An orchestrator, which in our example, is the Humanitec Platform Orchestrator.
- The CD system, which can be the Platform Orchestrator’s deployment pipeline capabilities — an external system triggered by the Orchestrator using a webhook, or a setup in tandem with GitOps operators like Argo CD.
Monitoring and Logging Plane
The integration of monitoring and logging systems varies greatly depending on the system. This plane, however, is not a focus of the reference architecture.
The security plane of the reference architecture is focused on the secrets management system. The secrets manager stores configuration information such as database passwords, API keys or TLS certificates the application needs at runtime. It allows the Platform Orchestrator to reference the secrets and inject them into the workloads dynamically.
The reference architecture sample implementations use the secrets store attached to the Humanitec SaaS system.
This plane is where the actual infrastructure exists, including clusters, databases, storage or DNS services. The configuration of the resources is managed by the Platform Orchestrator, which dynamically creates app and infrastructure configurations with every deployment and creates, updates or deletes dependent Resources as required.
When it comes to the assembly of these planes, integration is more intuitive for some layers than others. For example, platform teams are responsible for wiring individual plane components to each other, as well as binding one plane to the other. The raw end-to-end flows must also be tested and refined to ensure smooth integration and an optimal developer experience (DevEx).
How It Works
So, we know the benefits an enterprise-grade IDP can bring. We’ve identified the components comprising an IDP. And we’ve established the speedy MVP potential using the open source implementation code. But how does it all come together?
By using our OSS workload specification, Score, developers describe how their apps fit together and define which resources they depend on. The Humanitec Platform Orchestrator aligns these requests with the baseline configurations established by the platform team. With each git-push, the Platform Orchestrator interprets the configurations and resources needed for a workload to operate, generating application and infrastructure configurations based on the platform team’s predefined rules. Then, it executes them following the “Read” — “Match” — “Create” — “Deploy” pattern:
- Read: Interpret workload specification and context.
- Match: Identify the correct configuration baselines to create the application configurations and identify which resources to resolve or create based on the matching context.
- Create: Create application configurations; if necessary, create (infrastructure) resources, fetch credentials and inject credentials as secrets.
- Deploy: Deploy the workload into the target environment wired up to its dependencies.
One Step Closer to Building Your MVP
So there you have it. Your organization now has a way of building an effective enterprise-grade IDP in the fastest possible way. The new open source implementation codes tailored for AWS and GCP setups are an exciting new development for the platform engineering community and will save you hours in the design process.
Ready to get one step closer to an MVP of an effective IDP? Here’s how:
- For a head start on building your IDP, check our reference architectures:
- Expand your platform knowledge and explore our learning path.
Have fun building an IDP developers will love!