CI/CD / DevOps / Security / Sponsored / Contributed

Continuous Integration: The New Frontier for Security Pros

12 Jan 2021 1:00pm, by

GitLab sponsored this post.

Cindy Blake
Cindy is a senior security evangelist at GitLab.

In 1960, President John F. Kennedy famously introduced the “New Frontier” as his political slogan and legislative program, alluding to a new era of change and an upending of the status quo of a post-WWII America. The application development world is facing a similar, albeit less dramatic, new frontier as it embraces a shift left for DevOps.

Enterprises that embrace DevOps understand the need to bring security into the fold earlier and collaboratively — they recognize the need for a shift left to empower their developers to find and fix security flaws sooner in the DevOps lifecycle. Many of these organizations have done this by integrating their security scanners into their CI pipeline by using APIs and plug-ins, as well as CI scripts, to provide scan results to developers within popular CI tools like Jenkins and Circle CI. But, they may have chosen a path that trades one challenge for another.

There are two paths to choose from, one of which is less commonly known to security professionals. The common path is to cobble together the security team’s favorite security scanners to feed results into the CI pipeline for visibility by the developer, expecting the developers to adapt. The other path, better known by DevOps professionals and software engineers, is to choose a CI that embeds security scans out of the box — adapting security to fit the developer processes, without compromising efficacy or risk.

Organizations that have chosen to integrate individual security scanners into their pipeline will inevitably face at least one or more of these challenges along the way.

Maintaining a Fragile Integration 

First, integrations are very difficult to achieve and maintain. The DevSecOps Realities and Opportunities whitepaper, based on research conducted by 451 Research (commissioned by security firm Synopsys), shows that the most significant application security testing challenges inherent in respondent’s CI/CD workflows were a lack of automated, integrated security tools (61%). Synopsys points out in its blog article that while most application security tools have a command-line interface (CLI) to integrate the tool into the CI/CD pipeline, you must consider the multiple defined checkpoints in the pipeline where these application security tools run.

Illustrating this point, Synopsys explains:

“…when pre-commit checks are complete, the pipeline starts, and an incremental static application security test (SAST) is run. Once that test is complete, the pipeline should move seamlessly to the next phase: software composition analysis (SCA) and a deeper, more thorough SAST assessment.”

It shows SAST at pre-commit in the Web IDE, incremental SAST upon code commit, and another SAST scan at build. It also shows DAST, IAST and fuzzing later in the lifecycle during Test. Each of these points requires thoughtful scripting into your CI/CD pipeline, potentially with multiple tools if you use different tools for Source Code Management, CI and CD.

Add to this hot mess the complexity of each integration having its own unique requirements and release schedules. For example, one scanner may report results in PDF and XML formats, while another tool may produce reports in JASON or HTML. Unfortunately, this can be a huge challenge when it comes to customizing plugins for common activities like defect tracking, updating metrics dashboards, and breaking the build. You are in a constant state of maintaining a fragile toolchain.

The Security pro should consider what DevOps pros already know. A single platform with security integrating into CI/CD can not only eliminate this maintenance nightmare but actually provide the even greater benefit of efficiencies and reduced risk. Vulnerabilities can be eliminated at their source with insight from SAST, but also DAST and fuzzing, by using the review app already included in the CI function.

Providing the Developer with Actionable Findings

The UI experience may be different for each scanner that you wish to integrate. One may provide actions, another comments only. Comments only still require copying the information to another tool to create a work ticket (or issue), for instance. It may not have information like line-of-code, helpful to whoever is entrusted with the remediation effort. Even actionable results may not be automatically integrated with other tools like Jira, so therefore may require yet another point of integration to maintain.

Enforcing Required Scans and Policies 

Dev teams are known to work faster than security teams can keep up, likely because there will always be more developers than there are security pros. Security teams must rely on automation to apply security policies that reflect their organization’s unique risk appetite. If you have a complex chain of tools stitched together, it can be challenging to apply the proper security settings everywhere they are needed. Each scanner typically has its own settings, but then you must set the CI pipeline to apply what is acceptable (or not) to pass onto deployment. Often these settings must be replicated from project to project.

Visibility Across Tools 

According to SmartDraw’s 2019 DevOps Visibility Report, 84% of organizations say that DevOps visibility is important to them. Implementing an entire security toolchain into any existing pipeline can be challenging. Can you see, across DevOps tools, who changed what, where, when, and why? If someone grants an exception and approves a merge request/pull request in CI, can your security dashboard see who did this and why? Without end-to-end visibility throughout the development lifecycle, teams are challenged when confronted with audits and regulatory compliance.

Ability to Scale Across Hundreds of Projects

If you’ve set up a DIY CI pipeline for a project, congratulations. Now, let’s say you want to apply that technique to projects that use different programming languages. You’ll need a new pipeline. That’s one more pipeline to maintain. Like versions of spreadsheets or slides, how do you ensure they are kept in sync? What you really need is one CI template that automatically identifies the right scanner for the programming language used. The bigger an organization gets, the more complexity across projects.

Unpredictable Costs

If you are paying for security scanners by the scan or by the application — or worse, by the size of the scan — it can be difficult to predict costs until you get the bill. Factor in the complexity of iterative DevOps scans and a more decentralized scanning environment, where costs are determined by the actions of the developers. Now layer in a multiyear forecast. How many more scans, apps or GB will your scans require when DevOps practices are more fully deployed in your organization? You need a solution that charges by the seat, a much more stable and predictable variable. If you take a more holistic approach toward CI and security together, you can get there.

So how can you avoid these challenges, or at least minimize their occurrence?

A New Frontier: Embrace Security Automation Within CI

The key to a successful DevSecOps implementation that can scale is security automation embedded within your CI. Not only does this help eliminate security bottlenecks, but it also empowers developers and allows security to focus their energy and efforts on larger issues.

First, Consider a Single, Integrated Platform

If your developers are already using a CI platform that includes security, embrace it. See what it can do for you. If there are shortcomings, they may be easily overcome by a united effort between Dev and Sec. If they are not using a CI that already includes security, you may be able to convince them to make the change. Many of the leading stand-alone CI tools require complicated and fragile CI scripts, and many engineering shops are ready for a change. If they feel stuck with what they have, they can still use their incumbent CI tool as their CI of record, but do one single integration between the integrated platform with CI and security — like GitLab and its existing CI. That single CI to CI integration can be far easier to maintain than multiple security tool integrations to a legacy CI.

Not only can a single platform spare you integration headaches, but it can provide visibility for audits across the entire software development lifecycle. And by having a single source of truth to log changes, manage role-based access, and capture vulnerabilities, you can encourage transparency across shared tools and build trust between your developer and security teams.

Next, Harness CI to Apply Security Policies

Once you have determined what your policies are, simply configure them. Things to think about include:

  • Which scanners will you enable?
  • Which scanners will you require?
  • Will you require approval (from which people/groups?) for vulnerabilities that are critical or high severity?

Some vendors have made it easy to configure a pipeline with check-box UI’s to toggle on/off commonly used DevOps tools and Security policies with zero programming knowledge. These are usually sufficient unless you have unique requirements. If you do, the CI pipeline can be scripted pretty easily in the CI.yml file. Markdown language is really straightforward and is no more difficult to learn than a simple spreadsheet macro. The key difference between this path and the DIY path is that here you customize your pipeline once for a variety of languages and across all of your scanners (SAST, DAST, Dependencies, Containers, fuzzing, etc).

Most shops want to ensure that developers are indeed using the required scans and policy settings. The easiest way to do this is to templatize your CI settings. Administrators can set up CI templates and require their use for every project, or for specific groups of projects. With end-to-end visibility, if an authorized insider changed the template or even the policies for a specific job, it would be captured. Look for a CI solution like GitLab that has all templates you could need already baked into it, but also allows for customization to match whatever policy is needed. By using templates, you’ll simplify your pipeline process by having parameters already built-in and that ensure consistency.

It’s also a good idea to periodically assess the effectiveness of your existing policies. How often are they overridden? It may be time to use a CI solution that allows you to tailor your templates and permissions as needed. Look for tools that help your security team set guardrails without needing to understand how to program CI yml files, but also be sure to involve your devs in the tool selection or introduce ways to shift left that do not impose on their workflows and preferred toolsets.

While this New Frontier of DevSecOps and the shift left can feel like uncharted territory, by starting with the developer workflow, considering a single tool that unites Dev and Sec, and by harnessing CI, you can improve the efficiency of all involved while still reducing risks.

Learn more about DevSecOps at GitLab.

Synopsys and CircleCI are sponsors of The New Stack.

Feature image via Pixabay.

A newsletter digest of the week’s most important stories & analyses.