This post is the first of a two-part feature series on DevSecOps. Check back Thursday for the second installment.
The idea behind DevSecOps is little more than an extension of DevOps. Just as developers would, in a waterfall development style, throw projects over the wall to the operations team to make them work in production, even with “DevOps” security is handled completely separately from either the application development or the operations.
The term DevSecOps is often used in conjunction with “shifting security left.” This means the developer of the application should take security factors into consideration of their design, rather than leaving as a separate task later for the security team. While it does seem that most people naturally focus on how DevSecOps approaches focus on integrating security into the development process, especially the CI/CD pipeline, one could also argue that DevSecOps involves a “shift right” as well.
“It’s an umbrella term, it applies just like DevOps, both to how we build and sort of ship and deliver software, and how we sort of operate it in runtime,” said Guy Podjarny, founder of container security company Snyk.
Why Adopt DevSecOps?
Perhaps the relative neglect of how operations and security fit together is because the top benefit most people discussed from the move to DevSecOps were neither improved security nor improved reliability — traditional security and operations metrics. In most cases, the reason organizations seem to adopt DevSecOps has to do with getting applications to market faster — in other words, to meeting incentives typically associated with the developers on the DevSecOps team.
“I think the organizations that have successfully done DevSecOps, they are able to deliver things to market, new products, new features, much faster than their competitors,” explained Matt Chiodi, chief security officer for public cloud at Palo Alto Networks. “There are pure business benefits.”
In a traditional security set-up, security teams often don’t review code until it’s already gone through the entire development pipeline. When problems are discovered, the security team will send the project back to the developers with a list of things to fix. This long feedback loop slows down the project’s delivery while also heightening the tension between security and developers.
In practice, keeping security separate also makes security problems more likely to slip into production. “Most organizations spend 80% or more of their time, looking at the run phase,” Chiodi said. “When security is evenly distributed across build, ship and run, you end up with a much lower level of vulnerabilities that then surface in the runtime.”
Nonetheless, among the security professionals interviewed there seemed to be a consensus that improving the organization’s security posture is not the core motivation for adopting DevSecOps, but rather a happy secondary effect.
Getting DevSecOps Right
What organization would not want to deliver applications both faster and more securely? Many newer companies adopt a DevSecOps approach from the very beginning, but getting to successful DevSecOps is not easy for established, large organizations. Moving from DevOps to DevSecOps is probably at least as challenging as moving from waterfall to DevOps.
“The teams I have seen be really successful at DevSecOps have buy-in at the executive level,” explained Newcomer. DevSecOps requires a major cultural change, and involves shifting incentives for everyone involved. Without high-level buy-in, the kind of cultural change DevSecOps requires is perhaps impossible to make.
“Anytime you have to change the way people think or act, that’s going to be the hardest uphill battle,” explained Hillary Benson, head of product at container security company StackRox. “And there’s no real established blueprint for how you to get DevSecOps nirvana.”
Changing the Security Role
“Security has the reputation being the team of No,” Newcomer said. The goal of DevSecOps is to change that dynamic, so that security is no longer an obstacle on the path to deployment. But that involves changing how security professionals work.
First of all, DevSecOps requires a high level of automation. Security teams should not be running manual tests at all, instead tests should be integrated in the CI/CD pipeline. Feedback on an application or feature’s security risk should be available without intervention from a security professional.
Instead, in a DevSecOps model security teams’ role is to create security policies, help developers and operators understand the output from security automation tools and server as advisors. Developers do not need to become security experts, but they do need to become more knowledgable about security. Security professionals also undoubtedly have to become more familiar with both infrastructure and software development.
This isn’t always an easy path. “If you’re a security professional and you’ve spent a lot of time learning these skills, you’re probably not super excited about moving into something that makes you learn a whole different set of skills,” explained Benson.
One of the major challenges with DevSecOps is that it involves bringing together teams that are incentivized in different ways. Developers want to get new features out the door fast, security teams want every single potential security risk addressed, operators want super-reliable applications.
This tension is one of the core reasons that a successful DevSecOps transition has to have high-level buy-in. Even with security integrated into the DevOps teams, it’s important to have shared KPIs that bring together the traditional goals of everyone on the team.
But then who is ultimately responsible for security? That’s a sticky question — some people think security should still be the security professional’s responsibility, some think that responsibility should shift to developers, others that it should be everyone’s responsibility. The risk, of course, of “everyone’s responsibility” is that security becomes no one’s responsibility.
How Do You Measure Success?
Considering the motivations most companies have for adopting DevSecOps, it’s no surprise that that development velocity was a key success metric for DevSecOps. There’s another advantage using development metrics: Security is very hard to put numbers on.
“The difficult part of measuring anything in security, in particular, is that you’re sort of measuring the counterfactual,” explained Benson. “If if nothing gets breached, and it means you’re doing a good job, or it means you were lucky.”
But if DevSecOps is at least as much about cultural transformation as it is about technologies, it might make at least as much sense to measure success by organizational markers. For example: Are security professionals spending their time running tests themselves or creating/configuring tools for DevOps teams to use? Are security professionals aware of and taking into account the business rationale for features when they make recommendations? How frequently do development, security and operations talk to each other?
Palo Alto Security, Red Hat and Snyk are sponsors of The New Stack.