Part 2: The Secret to Winning IT Security Roulette

SaltStack sponsored this post.
In part one of this two-part series, I talked about how cybersecurity professionals oftentimes feel like they’re playing roulette with their infrastructure. For many in this field, the job can feel like a long night in the casino where the longer you stay, the more likely it is that you’re going to go home a loser. Sure, your company may be OK for now but you know deep down that it’s only a matter of time before you get hacked.
In part two, we take look at the differences between IT and Security roles and how we can better respect their responsibilities. I’ll also talk about the concept of SecOps and redefining Threat Intelligence to identify potential vulnerabilities in a different way.
Understanding Security and IT People

Who are these people and what drives them? The short answer is security and operations teams (DevOps, admins, SREs, etc.) have fundamentally different priorities, motivations and objectives. Security teams are all about visibility and understanding what’s in the deployment and infrastructure. The bulk of everything they do revolves around scans. Their expertise is centered around being able to identify explicit problems.
Subsequently, they end up with these incredible views of the infrastructure. Security teams start by gathering a significant amount of data and information. This data helps them understand if files have changed in ways they shouldn’t, illustrates if we’re in compliance with our hardening policies and shows new software vulnerabilities. All of this information gets spun into a bonanza of security alerts, and ultimately, their pinnacle is to file a bug report and ask operations to take care of it. It’s no different than taking a problem and throwing it over a fence for someone else to fix.
Operations people are the ones on the ground and in the trenches. They’re the people who are doing the maintenance of the systems. They’re also the people who are maintaining the cloud infrastructure and deploying the applications.
In short, security people find and identify issues. Operations people ensure applications are running and that new applications are deployed. Both teams exist to ensure that developers’ lives are easy, their code flows smoothly into production and that a site is reliable.
Do operations people care about security? Is security one of the most important things on their list in an incredible way? Absolutely, but are they going to be hurt more today by the infrastructure going down and an app not being deployed or by a security breach?
Let’s look at it through their eyes for a moment. The game of roulette is already being played because nobody knows when they’re going to be breached. When the breach does happen, where is the blame going to fall? Most likely, it’s going to land on the security executive and their teams. Operations motivations are not aligned with the security objectives of the organization! Where this problem really stems from is that the disconnect between the teams isn’t just about communication. In many ways, it’s a fundamental disconnect about how both sides of this issue think about their jobs, how they address the problems that they’re dealing with and how they deliver results.
When Something’s Not Working You Need to Try Something Else
Threat intelligence today is about identifying threats that can be breached in very specific ways. Generally speaking, intelligence is focused on a breach that can happen because the system is exposed to the internet.
Therefore, it’s about a particular vulnerability that is given more attention. However, when teams look at these breaches, many of them actually come from the inside. They are not fitting the profile and that’s something we should be afraid of.
Now that things are happening faster and in greater numbers, it doesn’t take long before you have a big pile of issues that you have to deal with. This app has to keep running, that app has to keep running, they all need to keep running… so what do we do?
We need to stop burdening our systems with security software. We need to be able to actually deploy remediations and fixes. We need to have an assurance that we have a hardened infrastructure. We need to be able to go back to a congressional committee when a major financial institution gets hacked and give them an answer along the lines of “we acted responsibly.” I think we can all agree this does not happen very often.
We need to fundamentally rethink how we are approaching this problem.
We need to rethink how everyone involved can operate in a different way than the traditional approach and focus on truly accomplishing security; not merely being aware. Sure, it does a lot of good to remove the fog of war so we can see what the enemy is doing but that’s only half the problem. We still need to be able to do something about it.
SecOps Is the Key
This is where the concept of SecOps comes in. Now, SecOps as a phrase has been tossed around a lot but we’ve had trouble in the industry getting SecOps to gain traction.
One of the main impediments to SecOps’ adoption is we’re still following that “scan and over the wall” approach. SecOps stems out of the philosophies of DevOps. When we go back to the roots of that movement, it originally had less to do with configuration-management systems and CI/CD pipelines (which is what DevOps has evolved into); rather, it has more to do with opening up inter-team communication. The question then becomes: How do we open up better team communication between operations and security? How do we make sure that our operations teams can enable security to make changes in a safe, governed way?
History has taught us that it’s very, very difficult to get human beings to talk to each other and get along in large groups; we all know this. In almost every industry we need tools to help facilitate even simple communication. We need to take a better approach that helps us change how these two teams communicate and work together. In many ways, it does need to be different from the DevOps movement.
Applying SecOps Best Practices to Reality
At SaltStack, we’ve begun to see effective SecOps workflows where security teams and operations teams are tied together. Security is gathering information, which is fed into a new SecOps management tool that we’ve created (plug, plug) called SaltStack SecOps. The goal is one platform that scans and remediates in a single cycle. We need to be able to take the scanning, compliance and vulnerability data and make sure the scan itself is accessible to the operations teams because they need to have that automation presented to them. Sure, there’s always going to be cases with nuances that are difficult — that’s life — but teams should have tools that can automate security fixes 80% of the time. In this way, the execution of those automation routines is delivered to the operations team and they won’t have to build them every time.
Some of you have already started cursing my name for saying that we should have tools that can automate the 80% case. But trust me; I know there are many situations where the best tool is not going to be able to magically present the absolute operational solution.
We also need to consolidate scanning tools. For people working in security, that may sound ridiculous and unattainable because there are so many different agents to do these scans.
However, with regards to compliance scans, many of the vendors have completely different agents — one may do a CIS compliance scan and another will do a STIG scan. They can’t even make one agent do the same thing. Organizations need to insist their security vendors are able to offer more powerful agents or viable agentless alternatives, that can provide combined scans and do it very quickly. Waiting anywhere from 24-hours to a week to execute a scan is no longer acceptable. Teams need to be able to kick off a scan and get something back in minutes or seconds. The security industry needs to look like an industry that focuses on the complete picture and how teams deliver secure infrastructure as opposed to focusing on the thing they seem to talk about the most: threat intelligence.
There are few things in my life that I find more annoying and unhelpful than when someone walks into my office and says: “Tom, there’s a problem over there” and walks out. If I have a highly effective employee, they’ll walk into my office and say: “We have a problem, here’s what we’re going to do about it.” There’s a discussion, some refinement, a little buy-in and then we go. How helpful is it really if a neighbor knocks on your door, tells you your house is on fire and leaves? Yet, that’s exactly what many security tools do. They don’t get a bucket or a hose, they just stand on the lawn and in a panicked stricken voice and yell: “Oh my gosh, you’re going to die!” We have enough talk. It’s time to look for security that acts.
Feature image from Pixabay.