Better Defense Against Spectre and Everything Else with Patching, SaltStack Hardening

In the last couple of years, we’ve seen an unprecedented rise in the number of reported security vulnerabilities affecting infrastructure software. According to common vulnerabilities and exposure (CVE) statistics issued by the National Vulnerability Database, 2017 saw more than twice as many vulnerabilities than reported in 2016 and 2018 has already seen roughly the same number of vulnerabilities revealed in 2016. Those numbers also don’t even include the nearly 8,000 vulnerabilities that weren’t assigned a CVE in 2017.

While 2017 saw its fair share of especially lethal vulnerabilities with the WannaCry and the Equifax hack among the most well known, 2018 started strong right out of the gate with the Spectre and Meltdown vulnerabilities.

In January, we shared our insight about how SaltStack can help remediate Spectre vulnerabilities. We still believe, as we did then, that intelligent, event-driven security automation is the key to assessing risk and mitigating threats for any given organization.

Here are a few ways to help secure your data against Spectre as well as against numerous other vulnerabilities, many of which have yet to be revealed.

Typical Vulnerability Response and Patch Cycle

Organizations typically employ a two-tiered approach when patching vulnerabilities within their infrastructure. The first tier usually involves vulnerabilities that are patched as part of a routine patch cycle; this process could take place weekly, monthly, quarterly or even yearly. (One true but terrifying fact is some organizations only patch yearly).

The second tier is reserved for extremely urgent, high-severity vulnerabilities such as WannaCRY, Apache Struts2/Equifax (CVE-2017-5638) or HeartBleed. When such vulnerabilities hit the news, the usual process to patch vulnerabilities takes a backseat. The mindset of “do whatever it takes to patch the vulnerability now” takes hold. This is especially true when C-Level executives ask for daily — if not hourly — updates on vulnerability-remediation status.

SaltStack can be invaluable to SecOps teams in both situations, but it’s really in the second scenario that its true value shines through.

In such a scenario, three urgent questions need to be answered:

• Are we vulnerable to [insert vulnerability here]?
• If so, how many assets are vulnerable?
• How soon can we patch, remediate or implement a workaround?

To explain how SaltStack can help, I’ll use two very well known vulnerabilities from recent history: HeartBleed and MS17-010 a.k.a WannaCRY.

Identifying and Patching Vulnerable Assets

The first step to patching vulnerabilities is identifying which assets are vulnerable. Vulnerabilities are typically tied to a specific version of the software or hardware; for example, v1.0 is vulnerable and v1.1 is patched. Step one of the process is to enumerate software versions of a vulnerable application installed in the infrastructure.

SaltStack ships with an extremely powerful remote execution and automation framework that normalizes routine system operations (such as listing software versions, and upgrading and removing software with a single command), which are referred to as execution modules.

Let’s take the example of HeartBleed and see how SaltStack would have helped.

HeartBleed

In the case of Heartbleed, the first question to be answered was: “How many assets have OpenSSL on them and which versions are installed?”

With Salt, that question can be answered with a single command:

Notice the system normalized the execution module and function “pkg.version,” which works across a wide variety of operating systems. SaltStack asks for the package version intelligently by calling rpm on Red Hat systems, apt-get on Ubuntu and related utilities for other operating systems.

This makes identifying vulnerable assets in tense situations easy. Output of the command can be formatted into YAML or json to perform additional processing on the output.

Here’s an example :

Patching OpenSSL

Just like listing OpenSSL versions, upgrading OpenSSL to the latest version can also be done with a single Salt command. Here’s an example :

[Note: on apt-based systems, this will perform a full system upgrade.]

That returns the following output after completion:

These commands execute and complete in a matter of seconds across thousands of systems. So, having access to a powerful automation platform such as SaltStack helps decrease the risk from a vulnerability that is actively being exploited in the wild from days to minutes.

In some scenarios, such as zero-day vulnerabilities when no patches are available, the only option is to implement a workaround or stop the service. SaltStack can be helpful in such scenarios as well. Here’s an example:

SaltStack ships with more than 475 execution modules such as pkg & service, and it’s easy to add new modules or extend existing modules to satisfy a specific use case.

WannaCry

When WannaCry ransomware broke out, it quickly infected a large number of systems by exploiting a vulnerability in the SMBv1.0 protocol. It leveraged the EternalBlue exploit, which was disclosed as part of the ShadowBrokers dump.

Disabling SMBv1.0

In response to WannaCry, one of the first recommendations to limit infections was to disable SMBv1.0 before a patch could be applied.

Just as in the previous case, SaltStack users could accomplish this by running a single Salt command against their Windows infrastructure using the reg and system Windows execution modules. Here’s an example:

If the registry fix was implemented through some other method, such as GPO or PowerShell, SaltStack could help verify the fix with reg read function.

Rebooting system

Setting the registry key was not sufficient though; for it to take effect, the systems needed to be rebooted as well. SaltStack orchestration can be applied to task automation to reboot systems as needed:

Applying Patches

And finally, when it’s time to apply the patch, Salt can download and install the patch:

System Hardening with SaltStack

Gartner predicts that ”through 2020, 95 percent of cloud security failures will be the customer’s fault.” While reactive patching of vulnerabilities is good, preventing the exploitation of vulnerabilities through system hardening is a prudent approach as well. In fact, Gartner also suggests “workloads that leverage the programmability of cloud infrastructure to improve security protection will suffer at least 60 percent fewer security incidents than those in traditional data centers.”

SaltStack can be used to make any system programmable across hybrid cloud environments. A best practice is to configure systems properly the first time and every time, and then automate self-healing routines if or when a system falls out of a hardened and secure state.

System hardening prevents exploitation of vulnerabilities even when a vulnerable version of the software is installed. Disabling or removing Apache from a system if it’s not used as a web server is one example. SaltStack can enforce proper configuration on a remote system to meet or exceed system hardening requirements through a concept called Salt State files.

State files allow users to define the desired “state” of the remote system in a YAML file. Here are some examples:

New vulnerabilities are announced almost every day, while the processes to patch them or implement workarounds are roughly the same. Firms must patch better, but it is also in their best interest to “harden their systems,” as described above.  While no system, by definition, is ever completely locked down, the double-pronged approach of correctly installing patches and updates while also using hardening techniques will go a long way to achieving your security goals.

SaltStack is a sponsor of The New Stack.

Feature image via Pixabay.

Mehul is a senior product manager for security at SaltStack where he is responsible for building solutions that help Salt users secure their infrastructure across cloud, on-prem, hybrid and now IoT platforms. Before joining SaltStack, Mehul led multiple vulnerability management,...

Read more from Mehul Revankar