What news from AWS re:Invent last week will have the most impact on you?
Amazon Q, an AI chatbot for explaining how AWS works.
Super-fast S3 Express storage.
New Graviton 4 processor instances.
Emily Freeman leaving AWS.
I don't use AWS, so none of this will affect me.
CI/CD / Operations / Platform Engineering

The 6 Pillars of Platform Engineering: Part 2 — CI/CD & VCS Pipeline

Platform team workflows and checklists for building a CI/CD and VCS pipeline into their platform.
Sep 21st, 2023 8:07am by
Featued image for: The 6 Pillars of Platform Engineering: Part 2 — CI/CD & VCS Pipeline
Image via Pixabay.

This guide outlines the workflows and steps for the six primary technical areas of developer experience in platform engineering. Published in six parts, part one introduced the series and focused on security. Part two will cover the application deployment pipeline. The other parts of the guide are listed below, and you can also download the full PDF version for the complete set of guidance, outlines and checklists.

  1. Security (includes introduction)
  2. Pipeline (VCS, CI/CD)
  3. Provisioning
  4. Connectivity
  5. Orchestration
  6. Observability (includes conclusion and next steps)

Platform Pillar 2: Pipeline

One of the first steps in any platform team’s journey is integrating with and potentially restructuring the software delivery pipeline. That means taking a detailed look at your organization’s version control systems (VCS) and continuous integration/continuous deployment (CI/CD) pipelines.

Many organizations have multiple VCS and CI/CD solutions in different maturity phases. These platforms also evolve over time, so a component-based API platform or catalog model is recommended to support future extensibility without compromising functionality or demanding regular refactoring.

In a cloud native model, infrastructure and configuration are managed as code, and therefore a VCS is required for this core function. Using a VCS and managing code provide the following benefits:

  • Consistency and standardization
  • Agility and speed
  • Scalability and flexibility
  • Configuration as documentation
  • Reusability and sharing
  • Disaster recovery and reproducibility
  • Debuggability and auditability
  • Compliance and security

VCS and CI/CD enable interaction and workflows across multiple infrastructure systems and platforms, which requires careful assessment of all the VCS and CI/CD requirements listed below.

Workflow: VCS and CI/CD

A typical VCS and CI/CD workflow should follow these five steps:

  1. Code: The developer commits code to the VCS and a task is automatically submitted to the pipeline.
  2. Validate: The CI/CD platform submits a request to your IdP for validation (AuthN and AuthZ).
  3. Response: If successful, the pipeline triggers tasks (e.g., test, build, deploy).
  4. Output: The output and/or artifacts are shared within platform components or with external systems for further processing.
  5. Operate: Security systems may be involved in post-run tasks, such as deprovisioning access credentials.

VCS and CI/CD pipeline flow

VCS and CI/CD Requirements Checklist

Successful VCS and CI/CD solutions should deliver:

  • A developer experience tailored to your team’s needs and modern efficiencies
  • Easy onboarding
  • A gentle learning curve with limited supplementary training needed (leveraging industry-standard tools)
  • Complete and accessible documentation
  • Support for pipeline as code
  • Platform agnosticism (API driven)
  • Embedded expected security controls (RBAC, auditing, etc.)
  • Support for automated configuration (infrastructure as code, runbooks)
  • Support for secrets management, identity and authorization platform integration
  • Encouragement and support for a large partner ecosystem with a broad set of enterprise technology integrations
  • Extended service footprint, with runners to delegate and isolate span of control
  • Enterprise support based on an SLA (e.g., 24/7/365)

Note: VCS and CI/CD systems may have more specific requirements not listed here.

As platform teams select and evolve their VCS and CI/CD solutions, they need to consider what this transformation means for existing/legacy provisioning practices, security and compliance. Teams should assume that building new platforms will affect existing practices, and they should work to identify, collaborate and coordinate change within the business.

Platform teams should also be forward-looking. VCS and CI/CD platforms are rapidly evolving to further abstract away the complexity of the CI/CD process from developers. HashiCorp looks to simplify these workflows for developers by providing a consistent way to deploy, manage and observe applications across multiple runtimes, including Kubernetes and serverless environments with HashiCorp Waypoint.

Stay tuned for our post on the third pillar of platform engineering: provisioning. Or download the full PDF version of The 6 Pillars of Platform Engineering for the complete set of guidance, outlines and checklists.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Pragma.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.