Tom explains the history, process, and effects of the Log4J vulnerability.
Featuring Tom Merritt.
Please SUBSCRIBE HERE.
A special thanks to all our supporters–without you, none of this would be possible.
Thanks to Kevin MacLeod of Incompetech.com for the theme music.
Thanks to Garrett Weinzierl for the logo!
Thanks to our mods, Kylde, Jack_Shid, KAPT_Kipper, and scottierowland on the subreddit
Send us email to [email protected]
Transcript:
I heard this log thing is the worst vulnerability of all time
But I don’t see it anywhere.
Do I need to worry?
Confused?
Don’t be
Let’s help you know a little more about the Log4J vulnerability.
What is Log4J?
It’s a Java logging library. Which doesn’t involve trees or timber in any way. Logging in this case means it uses the Java library to log system events. This is exceptionally useful. Logging is something any system needs to do for purposes of troubleshooting. You want to know what happened if there is a problem and a log is often the way you can know. What machine or user requested what resource at what time etc. Like when you get a “page not found” on the web. That kind of thing is logged so that admins can see what page was requested from what link you came etc. When you see a page not found error log4j is likely active logging that. Minecraft uses log4j to record user commands. That can be useful later to see which commands are being used how often. Especially fi there’s a glitch in the game they’re trying to run down. And another program might want to look at such a log itself and parse a line or two to see if a user already asked for that command and when.
Log4j is piece of code in the Apache system that does this. It’s open source and been around since January8, 2001. If you don’t know this already, developers don’t write every piece of code from scratch. They frequently re-use blocks of code that have been shown to be fit for a purpose. Log4J is a really big example of that. Need to log something? No need to write a logging tool, use Log4J. That’s not the controversial part of this story. There are thousands of components in various languages from various systems that are open source and widely used.
On November 24th, 2021 Alibaba’s Chen Zhajun discovered a remote code execution vulnerability in Log4j and responsibly and privately disclosed it to the Apache Software Foundation so they could fix it.
On December 6th Apache released Log4J 2.15.0 that addressed the vulnerability.
On December 9th a proof-of-concept of the exploit was published on GitHub.
On December 10th the vulnerability was assigned a CVE (Common Vulnerabilities and Exposures number) and ranked as high severity. Also on December 10th large scale attempts to exploit it were first detected most notably on Minecraft servers. That leads to national security agencies publishing severe alerts. And the next day December 11th, major cloud services like AWS publishing security bulletins.
So it’s patched right? What’s the big deal? However two things make this blow up in a way other similar vulnerabilities don’t. One, it is relatively easy to implement. And two the attack surface is gigantic because so many instances of log4j exist. Literally billions of devices run log4j.
Log4j is a really common component in software packages. It isn’t in one place. It’s everywhere. Sometimes a package doesn’t even list log4j as a dependency, but one of the dependencies in the package has log4j as its own dependency. So you have to know its there to patch it. And you might not know unless you dig in and look. For 80-percent of packages, log4j is more than one dependency deep.
Log4J is present in More than 35,863 Java packages in the Maven Central repository– where Java packages are collected to make available to devs– and those packages are spread across millions of pieces of software. Not all of those packages are updated yet either
It’s fun to embarrass big companies by naming them as vulnerable to log4shell but the fact is it would save time to list the servers out there that weren’t. All servers log and almost all servers use log4j to do it.
Google security estimates it will take years to patch log4shell across the entire ecosystem.
What is the vulnerability and how is its exploited?
The vulnerability is referred to as Log4shell. Here’s why it exists and how it works.
Log4J looks at requests sent to a server and interprets them so it knows how to make a log entry. The vulnerability was that you could construct the request so that when Log4j tried to interpret it, it ended up executing code. That’s partially by design.
Let’s say a Minecraft user is entering a command. It might be useful later to know what the real name of that user is. So why not have the log add that. To do that you could very easily have log4j take a quick look at your server directory, maybe an LDAP– look up the username and pull in the associated real name into the log. And no this isn’t a privacy violation this is all info already not eh server you’re just arranging it to make it useful.
There is something called JNDI, the Java Naming and Directory Interface. It’s a way for a java program to look up information. This can be useful for lots of reasons, including looking up local resources like storage etc and also is often used for looking up directory information as in our previous example.
Log4J allowed any lookup to be parsed. That ‘s where things went wrong. I wouldn’t guess it was designed that way. Just nobody thought about it being used that way so they didn’t put in a safeguard that limited the lookup. However somebody finally did think of that. If the user input being logged included a lookup to a malicious server, Log4J would resolve that lookup like any other lookup, and resolving it could mean that a malicious server didn’t send a name associated with a username but instead sent some Java code that when executed installed malware. A typical remote code execution attack.
Once you’re in you can do fairly typical stuff like install a reverse shell to control the server or add the server to a botnet or install ransomware.
Sending the code into log4j differs depending on the software. But say, in Minecraft, where chats are logged, you could type the malicious code into chat and compromise the server in some cases.
So What are the dangers?
Well, first It is actively being exploited.
Because it’s a component that is added to software it can lurk in programs without people knowing it’s there, possibly until bad actors discover it.
It’s easy to exploit and has a HUGE attack surface area. Lots of options for people just looking to find an opening.
How is it fixed?
The best solution is to patch. As I said Apache addressed the vulnerability and has updated the patch a couple times to make it super tight. The latest as of this writing was Log4j 2.17.0 for Java 8 and up.
Patching sounds easy but it can be difficult in some situations where an enterprise’s systems are so complex that patching is complicated or not all instances of log4j are even known. Remember log4j is not a program, it’s a software component. You can’t just run log4j’s update program. You have to know where it is. What software includes it. And depending on what piece of software uses it and how it implemented it, that changes how that software needs to be patched. You might need to just update your Apache server. Or you might need to ferret it out in every piece of software which might require a whole system update. Or you might need to just go in and find it manually and update it by hand. This doesn’t mean they shouldn’t patch, it just means it’s not as fast to patch in all cases as downloading an update and running it. As one example, Google has 500 engineers going through the company’s code looking for all instances and figuring out how to patch them.
Companies like IBM, Cisco, Oracle, VMWare and others are issuing their own patches for software they provide that uses the log4j component. But to eradicate it from the internet entirely means that every software developer has to know they used it and update then get it to their software distributors and system operators and eventually to end users.
And they have to start using the new version now. Google has issued tools to help developers from accidentally including an older and therefore vulnerable version of Log4J in their software.
And so. What do you do if you’re an end user. Because I mean if you’re a sysadmin you probably already know what you need to do. But for us end users the short version is the usual advice of stay patched so that you stay secure. If you are a little more of a tinkerer you can brush up on what to look for and see if you can manually update any instances of log4j on routers and NAS’s and such if your hardware maker hasn’t patched it yet. There are some mitigation efforts out there you might try too. But honestly most individual people won’t be targets and most people will get patched by their vendors. The exceptions will be DIYers, and users of older hardware especially that which is out of support. The main targets are corporations and governments and the like. But keep an eye out.
Whose responsibility was it.
I’ve heard some folks say this is the problem with open source software and characterize it as a bunch of volunteers, making it seem like folks who didn’t have time to keep it secure./ I disagree with that characterization. Developers can’t foresee every consequence of use and Log4j has been around for decades without anybody figuring out this vulnerability. Also log4j is part of the Apache Software Foundation which is a nonprofit 501(c)(3) corporation that has been in operation for 22 years with about 1000 members.
So that is the log4shell vulnerability in log4j.
It’s a huge problem for admins and developers. So keep them in your thoughts and maybe buy them a coffee or something. Possibly a beer. Or 7.
In other words. Thanks SysAdmins. And I hope you know a little more about the log4j vulnerability