Integrate Security and Compliance into Your CI/CD Pipeline
Cloud-native applications are best served by a CI/CD pipeline that’s hosted and managed in the cloud using cloud-native tools. It follows that these applications would also integrate security and compliance in a way that’s native to a cloud environment. That sounds like a no-brainer, but it has important implications for how you structure your pipeline — including the measures you take to secure and harden it. Hardening your CI/CD pipeline involves tracking changes end-to-end, which is a challenge when running containers in production.
How developers can easily get from local development to deployment on containers and Kubernetes using cloud-native tools like Skaffold, Minikube, and Cloud Build was the focus of the first in this three-part series about CI/CD on Google Cloud. These tools make it easy to get started and scale a CI/CD process that spans local and cloud environments. In part 2, let’s discuss what changes with security and compliance in the context of cloud-native applications, and some tools that help secure a CI/CD pipeline for these applications.
Modern Approach to Secure Software Delivery
In today’s multi-cloud world, developers and Ops teams have to learn to plan for security. The rapid pace of development means more code being written and more releases being made. This introduces more unknown dependencies and vulnerabilities at each step and leads to new challenges when securing containers. Manual security processes, such as traditional identity-based deployment control, rely on human operational knowledge and trust, and cannot scale to meet the needs of enterprises with automated build and release infrastructure.
“Developers building cloud-native applications on Kubernetes need security practices that are scalable, automated, and container-aware,” says Andrew Phillips, a Product Manager in Google Cloud Platform’s DevOps division. “Thinking about security and compliance and baking it into your pipelines early is a good way to ensure there’s some amount of standardization across teams.”
What’s needed is a modern approach to security that covers not just runtime security but even pipeline security. This would take advantage of a container image’s content-addressable nature and tie deploy-time security controls to runtime constructs. This new approach to security would automate security during development without slowing developers down while providing the centralized controls Ops needs to ensure everything deployed meets security requirements.
A modern CI/CD pipeline requires deeper visibility and greater control as it manages a faster and more dynamic software delivery cycle. New solutions such as Grafeas, Google Container Registry vulnerability scanning, Binary Authorization and Kritis deliver this visibility, control, and ultimately, more robust security. Let’s take a look at each of these tools and the role they play in a holistic strategy for container security.
1. Managing artifact metadata with Grafeas
Container security starts at the first step of the CI/CD process — when a developer checks in new code and an automated build is triggered by a tool like Google Cloud Build. From the first step to the last, metadata is generated that describes the components like container images, VM images, or JAR files being used, the repository, user and commit where code originates, and any known vulnerabilities. This metadata acts as the building block of an end-to-end security strategy and should be stored and analyzed centrally.
Grafeas is an open source metadata API that serves to centrally store, query, and retrieve all metadata produced across the software delivery pipeline.
“The biggest advantage of using Grafeas is that it acts as the source of truth and can be confidently relied on when debugging and troubleshooting,” Google’s Andrew Phillips said.
This confidence comes from the fact that Grafeas allows the admin to define granular access controls so that metadata, once created, is not modified. This source of truth is essential for root-cause analysis and can provide real-time visibility during live attacks.
Further, Grafeas enables the compliance that’s required for applications handling data in highly regulated industries like health care and financial services. Based on a pluggable architecture, Grafeas makes it easy to add new metadata producers and consumers like third-party security scanners and new build systems while still taking into account the metadata they generate.
In order to enforce compliance and policy-based security, having a centralized metadata store with access control is valuable. Grafeas is such as a store for metadata, putting metadata right where it belongs in the practice of container security and compliance: front and center.
2. Scanning for vulnerabilities with Container Registry Vulnerability scanning
The public Docker Hub works for sandbox environments, though teams will still need to exercise caution. However, when shipping containers in production a private registry like Google Container Registry (GCR), is indispensable. It helps teams to be proactive about security as it brings control over the container images that developers can use, and which ones eventually get deployed. Container security scanning is an essential feature of a container registry where container images are scanned against a list of CVEs (Common Vulnerabilities and Exposures).
Container Registry integrates with Cloud Build so you can perform vulnerability scanning as soon as a container image is pushed to GCR. Once the Container Analysis API is enabled, GCR Vulnerability scanning identifies known vulnerabilities in the container images, thus enforcing a shift-left approach to container security.
Further, GCR Vulnerability scanning can be integrated with your CI/CD system using Pub/Sub notifications or Google Cloud Functions (e.g. to automatically file bugs and track vulnerabilities when they are discovered). You can also take advantage of existing Cloud Security Command Center third-party tool integrations to detect and manage security issues. This is essential in a pipeline that’s hardened by numerous security tools that are being added and removed based on need.
Finally, Container Registry delivers detailed reports on the severity of vulnerabilities and provides real-time status on fixes available for these vulnerabilities.
Grafeas, when integrated with Container Registry, can record information about vulnerabilities as metadata. Additionally, a build tool like Cloud Build can inform Container Registry about the origin of code. Sharing vital security information across tools in the pipeline delivers a more comprehensive view of security data, and helps to secure the CI/CD pipeline.
3. Enforcing security policies with Kritis and Binary Authorization
Security data is churned out at every step of the CI/CD process: commits and build records, test results, automated or manual reviews and container image scanning. Organizations can make informed decisions on what can or cannot go into a production environment based on this information.
Kritis is an open source tool for Kubernetes that enforces deploy policies using metadata from Grafeas. Google Kubernetes Engine also provides a hosted implementation called Binary Authorization as part of its security best practices.
“Kritis enforces two kinds of policies on container images based on Grafeas metadata,” Dan Lorenc, a software engineering manager at Google who works on the Kritis project, said. “The first set of policies defines which container images should be signed. Next, Kritis enforces the policy that only containers with all the required signatures can be run in the cluster.”
Kritis provides a control loop to define which container images should be signed. For example, only sign container images with zero critical vulnerabilities inside of them. Consequently, every container image has PGP signatures attached to it which are stored as attestations in Grafeas. These attestations can be done by an individual from a QA or security team, or an automated process. Lorenc adds that “these signatures serve to establish a chain of trust for the binaries.” The binaries, in this case, are the container images themselves.
Kritis then enforces a deploy-time policy that only container images signed by the required authorities are allowed to run in your cluster. This type of deployment-time security check is essential when deploying thousands of times per day cross dozens of production environments. It helps codify operational requirements that tie production deployments to build and release time security controls, with automation that can scale.
In conclusion, security and compliance can be challenging when managing a CI/CD pipeline that centers around containers. However, by adopting the right approaches, DevOps teams can harden and secure their CI/CD pipeline. This can happen only by leveraging the power of metadata with a tool like Grafeas, employing container vulnerability scanning using a capable container registry, and enforcing deployment-time security checks with tools like Kritis and Binary Authorization. These ensure that as the pace of development and deployment increases, organizations can still achieve compliance, governance, standardization, and production-grade security for cloud-native applications.
