All articles

The Log4-shitstorm: everything you need to know

Henk-Jan Angerman ·

Log4j and Log4Shell

Will we make it to Christmas, and how?

In these dark days before Christmas, still surrounded by measures to contain the ongoing pandemic, yet another malicious vulnerability has emerged in ICT environments worldwide. The NCSC immediately gave this new vulnerability, CVE-2021-44228 or Log4Shell for short, its highest rating: 10 on a scale of 1 to 10, or HIGH/HIGH, meaning a serious likelihood of major damage. The vulnerability is embedded in Apache Log4j software, widely used by many software developers because of its benefits and ease of use. Unfortunately, this also means there is a very high likelihood that unauthorised users, hackers, can gain access to various systems without much effort, because it is not always known where Log4j is used or which version is active.

Immediately after discovery, a major alarm was raised worldwide because of the apparent ease of exploiting this vulnerability, its widespread presence, the fact that it had already been exploited and its potentially enormous impact on businesses and society. In addition to the NCSC's rating of 10, Germany has now issued a red alert, the internet is “exploding” with warnings and all major software developers are publishing responses and advice. Major names such as Microsoft, Oracle, RedHat, ElasticSearch and VMware, as well as Twitter, Amazon, Apple (iCloud) and Minecraft, have already indicated that they may be vulnerable and are on alert.

Action plan

Panic is never a good adviser, and there are now some slightly more reassuring reports. What matters first is a systematic approach, starting by mapping where the vulnerability may exist within your own organisation. Reports of activity exploiting it already go back to early December 2021, so checks must go beyond a current scan alone. As an example of a systematic approach, the NCSC provides a clear step-by-step plan with guidance and third-party tools. Various cybersecurity developers have also developed tools or add-ons to their applications to help organisations.

Better safe than sorry

Immediate action is the first requirement in this situation. Given the concrete risk of intrusion and its consequences, closing all external access to potentially vulnerable systems may be a good first step, even if it reduces productivity or service delivery. Once potentially affected systems have been inventoried, access can be restored gradually. Many software developers now offer either a solution through an update or upgrade, or a workaround.

Silent killer

It has now been discovered worldwide that hackers in many places quickly used this Apache Log4j vulnerability to gain access to various systems, although this has not yet resulted in actual outages, system hijacking or data theft. Experts therefore expect that this access has mainly been used to install malware and ransomware that will remain in the background for now and can, and will, be activated later. It is therefore crucial to check for this too, detect every deviation from standards or normal system behaviour immediately and take action to prevent worse outcomes. Anti-malware and anti-ransomware solutions are more important than ever. Recent system and data backups must also be checked, as they could already be infected, affecting whether they can provide a safe fallback.

  • Investigate where Apache Log4j is used within your organisation and therefore where potential vulnerabilities may exist, including through a vulnerability scan
  • Upgrade to version 2.16.x (Java 8 required). Note: 2.15.0-rc1 is not sufficient. Alternatively, upgrade the software or appliance. For older Log4j versions, you can rename/remove JndiLookup.class. Note that any project still using log4x 1.x has an outdated, unsupported version with other known vulnerabilities: CVE-2019-17571 has a CVSS score of 9.8(!)
  • If you cannot upgrade Log4j, you can reduce the impact of the RCE vulnerability by setting log4j2.formatMsgNoLookups to True -Dlog4j2.formatMsgNoLookups=true on the JVM command line, but only for versions >= 2.10.0. Users of Log4j 2.10 or later can add -Dlog4j2.formatMsgNoLookups=true as a command-line option or add log4j2.formatMsgNoLookups=true to a log4j2.component.properties file on the classpath to prevent lookups in the log event message. Users of Log4j 2.7 can specify %m{nolookups} in the PatternLayout configuration to prevent lookups in the log event message. Remove the JndiLookup and JndiManager classes from the log4j-core JAR. If JndiManager is removed, JndiContextSelector and JMSAppender will stop working.
  • Use the NCSC Log4j GitHub repository in your investigation and scans as a source of IOCs, links to scanning options and lists of vendors and products or software that are or are not vulnerable. See https://github.com/NCSC-NL/log4shell for the lists
  • Apply the latest version (2.16.0), which prevents these issues, to existing installations immediately. If this is not possible for any reason, follow the Apache Foundation's advice at https://logging.apache.org/log4j/2.x/security.html (patching)
  • Check or install adequate security on all affected server systems to protect against ransomware and other threats (secure)
  • Monitor all server systems for unusual behaviour (threat hunting), particularly outgoing connections

Context: who and what

  • Impact: arbitrary code execution. Code can be retrieved from the public internet, or use LOLBins already present on the system, or simply retrieve shared secrets or environment variables and return them to the attacker
  • Targets: servers and clients running Java and logging anything through the Log4j framework. Primarily a server-side issue, but any vulnerable endpoint can be a target or pivot point
  • Downstream projects: until proven otherwise, assume anything that includes Log4j, or depends on something that does, is affected in a way requiring mitigation; see below
  • Affected versions: Log4j 2.x confirmed; Log4j 1.x only indirectly, through previous information disclosure vulnerabilities that are harder to exploit. The presence of 1.x is still not good: 1.x reached EOL in August 2015!
  • Devices: do not forget appliances, other systems or third-party systems that may use Java server components but will not be detected by unauthenticated vulnerability scans or simple exploitation tests, owing to lack of access and so on
  • Log forwarding: logging infrastructure often has many “northbound” (send my logs to someone) and “southbound” (receive logs from someone) forwarding or relay topologies. Chaining these together for exploitation must also be considered.
  • Cloud: several major providers are also affected

Scope / severity

  • “People are comparing #log4shell to ‘as bad as Heartbleed’. That is not right; it is much, much worse. Apart from having RCE as the impact, the number of interdependencies around Log4j, especially given its age, is orders of magnitude greater.” – @caseyjohnellis
  • “What people seem to be missing: the #Log4Shell vulnerability is not just an RCE zero-day. It is a vulnerability that causes hundreds and thousands of zero-days in all kinds of software products. It is a zero-day cluster bomb.” –@cyb3rops (Florian Roth)
  • “A project with a footprint like Log4j cannot be avoided as a transitive dependency, even if you do not import it directly. Log4j is a canonical logging utility for a huge ecosystem. Its current scope goes beyond doing due diligence.” – @rakyll (AWS)
  • The Wikipedia article on Log4j is informative for understanding its use and scope (https://en.wikipedia.org/wiki/Log4j)
  • Earliest known detection: 2021-12-01 04:36:50 UTC
  • Misnomers: no, it is not called LogJam either. That name is already in use. The first LunaSec post used that name, then chose a new one once they realised.
  • Pronunciation: the principal author pronounces it “log four jay”, not “logforge”

Important information and guidance:

Back to all articles