An Osquery Field Guide for Log4J Defenders

The Log4Shell vulnerability has quickly become one of the most severe vulnerabilities of the last decade. Since public disclosure, defenders have placed countless hours into incident response and remediation. We at Uptycs recently published a blog post on some useful osquery tables to be aware of to help with the investigation, remediation and/or exploit detection cycle for combating the Log4j vulnerability.
In this post, we are sharing a set of queries that work on the open source version of the osquery operating system query tool as well as other helpful validation and remediation techniques.
Log4Shell Brief
The current vulnerability is particularly dangerous since an affected application does not need to be internet-facing; as long as the user-provided input makes its way to the vulnerable application and is processed by the logging library, the application is vulnerable. This means that a well-crafted string sent as part of the HTTP header(s) or HTTP body could allow an attacker to gain remote code execution (RCE). RCE can allow attackers full command and control if they successfully compromise the application.
So which libraries are vulnerable and which version has the fix?
Vulnerable:
- log4j (1.x) is a legacy library. Even though it has its own vulnerabilities, none are as severe as this.
- log4j-core jar file from log4j2 library has the vulnerable class.
- log4j-core-2.0-beta9.jar or later versions are vulnerable.
Most updated version:
- log4j-core-2.12.4.jar (for JDK7) and log4j-core-2.17.1.jar (for JDK8 and later)
What Inventory and Information Should I Gather?
When it comes to understanding what assets have been potentially affected, we’ll need to quickly gather information and inventory of the environment. We suggest investigating development, staging, test, and production machines as well as Cloud VMs, containers and serverless applications. To further extend the power of osquery, defenders can also use cloudquery and kubequery, two open source tools that Uptycs maintains to fetch relevant data from cloud infrastructure and Kubernetes clusters.
Other assets that should be considered are internal registries/repos, containers registered, blob stores with build artifacts, and employee machines.
What Should I Look for?
Within the aforementioned instances, we suggest looking for information regarding vendor software with Java, vulnerable Java applications, and all artifacts related to log4j version 1 and 2.
Java Processes Running on the Host or in Containers on the Host
1 2 3 |
SELECT * FROM processes WHERE name LIKE 'java%' |
Affected JDK/JRE Versions
1 2 3 4 5 6 7 8 9 10 |
SELECT * FROM deb_packages WHERE name LIKE '%jdk%' OR name LIKE '%jre%'; SELECT * FROM rpm_packages WHERE name LIKE '%jdk%' OR name LIKE '%jre%'; |
formatMsgNoLookups=true
Note that changing this does not completely fix the vulnerability. Log4j-core should be upgraded to 2.17.1.
Processes with JVM property -Dlog4j2.formatMsgNoLookups=true
1 2 3 |
SELECT * FROM processes WHERE cmdline LIKE '%-Dlog4j2.formatMsgNoLookups=%' |
Processes with environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS=true
1 2 3 |
SELECT * FROM process_envs WHERE key = 'LOG4J_FORMAT_MSG_NO_LOOKUPS' |
Get all docker containers with LOG4J_FORMAT_MSG_NO_LOOKUPS environment variable set
1 2 3 |
SELECT * FROM docker_containers WHERE env_variables like '%LOG4J_FORMAT_MSG_NO_LOOKUPS%' |
Looking for Vulnerable log4j2-core
Processes with vulnerable log4j2-core in command line
1 2 3 4 |
SELECT * FROM processes WHERE cmdline LIKE '%log4j-core%' AND cmdline NOT LIKE '%log4j-core-2.17.1%' |
Processes with vulnerable log4j2-core opened by Java process
1 2 3 4 5 6 7 |
SELECT * FROM process_open_files o JOIN processes p USING (pid) WHERE <a href="http://p.name/"><span class="s1">p.name</span></a> LIKE 'java%' AND o.path LIKE '%log4j-core%' AND o.path NOT LIKE '%log4j-core-2.17.1%' |
Look for JndiLookup in All Open Jar/War/Ear Files.
Note that this can be an expensive query depending on how many files are open. Also, this can check for jars/uber jars/shaded jars but does not work correctly when checking log4j-core-2.17.1 jar.
1 2 3 4 5 6 7 8 |
SELECT * FROM yara WHERE count > 0 AND sigrule = 'rule class { strings: $cls = "JndiLookup" condition: $cls }' AND path IN (SELECT path FROM process_open_files WHERE path LIKE '%._ar') AND path NOT LIKE '%log4j-core-2.17.1%' |
More Event Tables
You can use the following event tables to help identify activity related to Java processes post exploit.
-- process_file_events
-- process_events
-- socket_events
How Do I Validate My Fixes?
Validating That JndiLookup.class Was Removed
You’ll want to make sure that this class is no longer loaded by your application.
For each Java process, if jps is available:
1 |
for i in `jps | grep -v Jps | awk ‘{print $1}’`; do jcmd $i GC.class_histogram | grep -q JndiLookup && { echo “PID $i MIGHT be vulnerable”; }; done |
Validating Version Upgrade
You may also want to validate that you have properly upgraded to a fixed version. To do so, you can:
- Get inventory of all jar/war/war/jmod files opened by the process.
- For each file opened by the process, check if it is or contains vulnerable log4j2-core jar.
- Validate on an on-going basis to avoid deploying vulnerable applications due to auto scaling, old build artifacts, etc.
There’s plenty more to be done after gathering inventory of your assets. Other security teams should look to implement a WAF to block jndi or ctx strings in user input. EDR/XDR teams should look to monitor processes spawned by Java processes and monitor egress traffic.
Uptycs Threat Research team recently released some research regarding the aftermath of log4j and continues to work feverishly to provide further observations on exploit.