Real-Time SBOM Focuses Supply Chain Security on Real Threats
A Software Bill of Materials (SBOM) is an inventory of libraries used in software and is typically generated at build time. The “chain” in software supply chain is there because build is just one step of many.
It turns out that closing the loop between build and production is critical to efficient vulnerability management in addition to securing the end-to-end supply chain. At EdgeBit, we call this loop the “real-time SBOM” — a report of which software and libraries are in use right now, powered by eBPF and other kernel APIs.
Useless Investigation Is an Expensive Problem
Automation is key for security, but security teams aren’t experts in each app, so “open a ticket for any vulnerability found” is a typical workflow. These tickets are a firehose – “shift left” is about moving security closer to the engineer, but it’s noisy and frustrating.
Here’s a few stats that show how it really adds up:
- It takes ~51mins to triage, fix and patch a single vulnerability
- A large enterprise is tracking over a million known vulnerabilities
If we apply knowledge about how our code executes back into our build pipeline, we can investigate real security issues related to executing code and deprioritize anything related to dormant code.
Inventory Is Harder Than You Think, Especially for Enterprises
Understanding where software is running is surprisingly hard for enterprises with many app teams, different clustering technologies, and multiple cloud environments. It’s never as simple as “just go look at the Kubernetes cluster” — who has just one? — or “go look in AWS” — which product?
Even harder is understanding if you have a specific dependency or library in use. Log4j is an example that everyone likely experienced themselves. If five teams confirmed they updated the library, have all five rolled out the changes? Is there an app we missed? When the next critical bug is found, hours and minutes matter if it’s being actively exploited.
How a Real-Time SBOM Works
By understanding which dependencies in your app are actually executing, EdgeBit can focus vulnerability investigation on those items and lower the priority of others.
A thought experiment for you — what percentage of your dependencies are live and in use in your production environment? Can you even begin to guess? Is it closer to 10% or 90%? The results are always surprising to engineers.
The tricky part of getting a complete inventory of a running system is 1) it’s highly dynamic and 2) we need to cut across sandboxing abstractions in order to get a complete list of packages installed inside and outside of containers. This methodology allows us to address a range of use cases no matter how your app is packaged or deployed.
By hooking into the Linux kernel, it’s possible to see what files and packages are being loaded and executed which starts to build a list of what’s in use. This data is sent to EdgeBit’s SaaS service to match the data with your build time SBOMs, vulnerability databases and the view from the rest of your server fleet.
Under the hood, a number of open source projects are utilized:
- edgebit-agent: installs eBPF probes and hooks into kernel
- syft: SBOM generation
- grype: vulnerability matching
- OSV: vulnerability database
- GitHub advisories: vulnerability database
Trio of Supply Chain Artifacts
Beyond the SBOM, there are two other supply chain artifacts you might not be familiar with:
- Vulnerability Disclosure Report (VDR)
- Vulnerability Exploitability eXchange (VEX)
It’s a pretty simple concept: alongside your SBOM, show that you’ve scanned and understand the known vulnerabilities attached to each software dependency (VDR) and if those vulnerabilities are not relevant based on how your software uses a library, communicate that (VEX).
NIST guidelines suggest both generating (and remediating) the issues found and communicating or sharing the report with your customers. Communicating the security state of your packaged software or SaaS service is a pretty radical change for some organizations.
Vulnerability Management Is Now a Multiplayer Game
Before the recent software supply chain executive order from the Biden administration, security was mostly a single-player game. A company tracked its own bugs, fixed them on its own servers with its own engineers.
Now, it’s mandated to be a multiplayer game. All of this activity needs to be published to the government (if you sell to them), to upstream/downstream vendors you partner with and directly provided to commercial customers.
If you use open source software or package another vendor’s software as part of a product, you need to track, understand and communicate the aggregated list of what makes up your ultimate product release.
NIST requirements outlined in SP 800-161 R1 requirement RA-5 cover how to share SBOMs and VDRs:
“Enterprises should consider publishing the VDR within a secure portal available to customers and signing the VDR with a trusted, verifiable, private key that includes a timestamp indicating the date and time of the VDR signature and associated VDR.”
If you remember our statistics from above about tracking over a million vulnerabilities… there is no better reason to start reducing that backlog than having to share it with all of your customers! Using this trio of supply chain artifacts will help security teams understand the full state of a piece of software and help cut through the noise that may be present by just looking at an SBOM alone.
Supply Chain Requirements Cascade to the Entire Industry
I believe that it’s just a matter of time before the same supply chain requirements are included in SOC2 and ISO 27001 compliance regulations. This change would cover both packaged software and SaaS services.
How is an SBOM+VDR+VEX useful for a SaaS? When the next critical issue is found, customers won’t have to file tickets and watch blog posts for news about a fix. Instead, they can use a machine-readable or human-readable report that’s automatically generated and shared. There’s a new number of practical considerations about how much to share and when to be worked out, of course.
The increase in transparency will be a net positive for the security of the entire software industry and open source communities, but it’s going to require an open mind and a cultural shift for some organizations.
