Prisma Cloud by Palo Alto Networks sponsored this article for The New Stack.
Cyberattackers look at things differently than the rest of us. So is it really any surprise that what a DevOps engineer sees as a straightforward piece of Infrastructure as Code (IaC), a determined hacker views as a route planner right to the heart of an organization’s systems?
According to Matt Johnson, developer advocate for Bridgecrew by Prisma Cloud, codifying infrastructure using tools like AWS CloudFormation or open source Terraform means you end up with reusable definitions of infrastructure — a massive benefit when building, deploying and managing complex systems.
The problem is those definitions can include misconfigurations, security flaws, even hard-coded credentials, which will be recycled and redeployed each time you use that code.
Moreover, as Johnson told The New Stack, it’s a cliché that “if a developer needs to do something, they turn to Google, they find a solution on Stack Exchange or Stack Overflow and then that gets copied and pasted into their codebase.” This means other people’s misconfigurations and vulnerabilities are now their misconfigs, too.
The problem is partly down to the fact that many of the publicly available IaC templates were originally intended as examples or tutorials, “to get people to ‘Hello World,’” as Johnson put it. If they’d been produced with all the relevant security elements, he added, they’d be 100 times longer, which would make them far less digestible as a learning aid.
‘It’s Definitely a User Issue’
Bridgecrew, the security platform company that was acquired by Palo Alto Networks last year, has scanned popular Terraform registries and the Cloud Native Computing Foundation’s artifacthub.io using its Checkov tool, and found “major issues” with more than 50% of templates or artifacts.
“The reality is you are more likely to download an insecure-by-default piece of infrastructure as code than a secure-by-default,” Johnson said.
It’s important to remember that the problem — and the ultimate responsibility for tackling it — doesn’t lie with the vendors. Amazon Web Services, for example, offers tooling to help customers detect whether IaC changes can cause problems, including AWS CloudFormation Hooks and AWS CloudFormation Drift Detection.
“It’s definitely a user issue,” Johnson said. “Terraform or AWS CloudFormation are not the things that are insecure. They are a set of tools that can be used or misused.”
But if users might be alarmed to find that they’ve incorporated insecure IaC into their own systems, their discomfort will shoot up even higher when they realize that the very ubiquity of that code means attackers potentially have a route map right through their systems.
“If you’re using one of those modules,” Johnson said, “I can see exactly what is and isn’t configured within that module.”
This means that as well as gaining an insight into possible entry points, attackers can map out potential lateral movements in advance, prioritizing where they should move next, deciding the best exploits to deploy — and all with a good idea of the targets they can reach.
“The more holes you have, the more likely you are to find a path all the way through,” he said — and misconfigs can give a direct route to the underlying system.
Avoiding public IaC templates is not a solution in itself. Users are perfectly capable of adding their own flaws — even when they’re not flaws at the time. “A secure default this year might be an insecure default next year, because of a change in encryption type,” Johnson noted.
Surfacing IaC Recommendations
DevOps — or should that be DevSecOps — has already tackled the problem of developers “forgetting” to run security tests on code by automating testing, and having those tests stored as code. So, when it comes to configuration code, said Johnson, the obvious thing is to have automated tools to do the same thing.
But “historically, security tools have not been built to be operated by engineers and developers,” he added. “They’ve been designed to be understood by security people, who are a different breed entirely.”
There’s a further hurdle, he said. Scan a typical modern organization’s infrastructure, and it will identify hundreds if not thousands of Common Vulnerabilities and Exposures (CVEs) — and, in parallel, hundreds if not thousands of Infrastructure as Code recommendations. But this can leave developers paralyzed as to what to tackle first.
So the challenge is to translate that information into something palatable to a developer and to do so without forcing the developer to leave their tools or environment or workflow. “Let’s try and surface that information as early as possible with a useful suggestion as to what they can do to remediate this,” Johnson said.
Bridgecrew’s Checkov scans for and identifies misconfigs in IaC code, including CloudFormation and Terraform, as well as other file types including Kubernetes and Docker. But with the acquisition of Bridgecrew by Palo Alto Networks, the technology has been integrated into Prisma Cloud’s cloud native application protection platform, which means Checkov’s insights can be put into a broader context along with other vulnerability information.
“The big new feature that we’ve been working on is this idea of supply chain graphing,” Johnson said. This builds a graph of which CVEs are in which containers, and which is deployed by which piece of infrastructure as code. “So we can actually visualize where there is a tangible attack path or not.”
For example, if a CVE is running in an application that is not connected to the internet, the chances are that’s not going to be as high a priority as a CVE running on a public load balancer allowing full internet access.
And this analysis needs to be extended to developers’ own toolchains, which themselves often rely on publicly available components, said Johnson: “We’re seeing more and more people trying to attack the CI/CD pipeline itself.”
After all, he added, “Why try and attack something in production if you can inject your own code into the developer’s system that then makes it into production for you?”
So, Palo Alto Networks’ platform now includes additional checks and policies to check how developers’ pipelines are configured, Johnson said: “Because that pipeline configuration is itself codified, we can go through that and say ‘Hey, you probably should turn on at least two approvals from internal before someone can merge.’”
It’s hard to imagine developers retreating back to the walled gardens they used to work in. The benefits of automation, IaC and reusability are too great, even if its associated dangers are becoming more obvious.
This means that automation is the only way to solve this problem, Johnson said. But that automation has to alert developers as well as security personnel and give them a clear indication of the risks they face rather than simply saying no. “As a developer,” he said, “that’s what I’d want.”
The New Stack is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.
Amazon Web Services and The Cloud Native Computing Foundation are sponsors of The New Stack.
Featured image by Radek Kilijanek on Unsplash.