News for hackers. Stuff that matters.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by jmaker on Wednesday January 17, 2024 @06:24AM (#39026320)
As long as the deps haven’t all moved to the JDK without Unsafe, I think the ecosystem is tainted by such deps for the foreseeable future.

In case you guys conducted an ecosystem survey, where are we at now? Lots are still running on JDK 1.8, some important deployments are on 1.6 or 1.7, even more are on JDK 11. The transition to JDK 17 has started only recently.

How much more hesitant will this make projects to upgrade the JDK? There’s quite some tearing already due to 1.8, 11, and now 17, particularly with Pivotal EOL support for Spring 2.7 release train. I wish we could wait with it till the transition to 17 is complete.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Wednesday January 17, 2024 @07:07AM (#39026629)
The JEP says you'll be able to use Unsafe at least until JDK 25. There's surely enough time for libraries to migrate away from unsafe by the time a significant number of the ecosystem is on 25. I don't see why this should have any impact at all on adopting 17 or 21, which are unaffected.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pjmlp on Wednesday January 17, 2024 @07:16AM (#39026689)
I hope the current state of large organizations staying in Java 8 as long as they can to avoid dealing with modules, is somehow a lesson for how to deliver this change.

Two months ago I did a fresh deploy of Java 8 in production for a SaaS solution, exactly because that is what the product and support team care about.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Wednesday January 17, 2024 @07:28AM (#39026796)
Staying on 8 has nothing to do with "dealing with modules" (JDK 11 had the exact same accessibility as JDK 8; the JDK's encapsulation wasn't turned on until JDK 16). The reason some need to stay on 8 is because they're using old, non-portable libraries that rely on internals that have since changed. Modularising the JDK has only helped prevent that from reoccurring, and has made version upgrades easier than they've ever been. Not modularising the JDK would not have helped people migrate one iota. The internal changes that broke the old non-portable libraries would still have broken them, only there would have been nothing to prevent this from happening over and over again.

Also, there isn't much new code being written for 8, and there's a whole lot more of Java code yet to be written than the non-portable code still on 8. In fact, even most existing projects are no longer on 8. Sure, if you only look at the pain, there has certainly been some. But if you look at the pain compared to the benefit, the lesson is that it's been a great success, especially compared to other languages (or even frameworks) doing similar significant changes. I think the vast majority of those who've upgraded consider it worthwhile (again, comparing cost/benefit).

Virtual threads, FFM, and the upcoming Valhalla and Leyden are all things that could not have been done without modularisation, and certainly not without significant internal JDK changes, which would have kept the same people on 8 even if there was some way (doubtful and definitely much more expensive) to achieve that without modularisation. We would have been in a far worse place without it: fewer new features and no end to the problem of migration.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by jmaker on Wednesday January 17, 2024 @03:37PM (#39033106)
While I do totally agree on the necessity of the steps you outlined, I have yet to discover a reliable ETA for Valhalla and Leyden, and Panama, it’s been like a carrot on a stick so far. And I do totally realize that certain consistency bounds must be in place to smoothen the future evolution of JDK, yet the chasm is still there and a quite significant one. I’m sure you are aware. There’s so many things going on at the same time to be aware of in Javaland, it’s getting quite tedious to persuade everyone to keep it up and not jump ship, after getting burned by the previous JDK (and Spring) upgrades.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Thursday January 18, 2024 @11:33AM (#39043835)
Panama was delivered in JDK 19 and is final in JDK 23, out this March.

We don't give ETAs as people may make business decisions based on them. As a matter of policy, we only announce a targeted JDK within the 6 months prior to its release.

> it’s getting quite tedious to persuade everyone to keep it up and not jump ship, after getting burned by the previous JDK (and Spring) upgrades.

If nothing else, they should be placated by the fact that upgrades are significantly worse in every other ecosystem.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by iscoelho on Thursday January 18, 2024 @01:10PM (#39045252)
> If nothing else, they should be placated by the fact that upgrades are significantly worse in every other ecosystem.

A lot of ecosystems care about their users and care about compatibility. Java is a toxic relationship at this point.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Thursday January 18, 2024 @02:29PM (#39046412)
I think that Java's record on compatibility is unmatched, and users are extremely happy with all the enhancements we've been delivering, which why Java's popularity remains sky-high. But no product can satisfy everyone forever, and if you're unhappy, we'd like to try and address the issues you have; if we can't, we'd be sad to see you go, but we certainly don't wish to hold you hostage.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by jmaker on Thursday January 18, 2024 @03:05PM (#39046903)
Yes, absolutely, Java’s compatibility story has been stellar. To the point that people got spoiled by it and no longer want to join the more agile release cycle. I’ve experienced folks being more like “oh finally, now I can express myself at least a bit like I got used to with some other language”, so not sure the enhancements you’re mentioning go as far as to put Java apart from its modern day alternatives, at least some of those concepts finally trickled through though. Still, a lot of Java deployments are deliver once, never touch it again, those users would be happy to never upgrade anything. It’s no fun to revalidate the semantics in a large project.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Thursday January 18, 2024 @03:19PM (#39047079)
> those users would be happy to never upgrade anything. It’s no fun to revalidate the semantics in a large project.

Yep, that's why we introduced the LTS service. People may not have noticed before we did away with major releases and changed the version naming scheme, but in the past you had no choice but to upgrade to semi-annual releases with major VM changes, like 8u20 or 7u4 (probably because the name was too similar to patch releases, like 8u30 or 7u5). Now, with LTS, we try to only backport security patches and major bug fixes to make sure that such legacy users get the stability they need.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by jmaker on Thursday January 18, 2024 @02:52PM (#39046719)
Gradle is still at JDK 21, albeit not comprehensively as it sometimes still fails to build on anything more recent than JDK 20. Most projects haven’t transitioned to JDK 17 yet. There’s a gap already. I don’t really want to draw parallels but Scala 2 to 3 wasn’t supposed to go the way Python had gone, everyone was certain that wouldn’t be the case, while .NET made a clear cut and somehow managed to move everyone forward, perhaps because of the monocentric organization of the ecosystem. From my experience, I can’t relate with your last sentence however. I do totally appreciate your efforts and investment into the Java community Ron, always have, but your words now feel like a disconnect, or perhaps I am simply no longer part of your target audience.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Thursday January 18, 2024 @03:04PM (#39046889)
I don't understand what you're trying to say. Python's migration has clearly been worse and took longer, .NET break their users every 6-7 years on average requiring significant rewrites (except .NET's users have been used to that pain for many years and accept it clear-eyed), and the JS ecosystem is such a mess that people rewrite their programs every couple of years because their libraries keep being abandoned or make incompatible changes as a matter of habit.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by iscoelho on Wednesday January 17, 2024 @05:58PM (#39034816)
No. JDK9/JDK11's adoption issues were due to --add-opens/--add-exports and the JDK developers hostile approach towards reflection.

It's an obsession over "pure" code that has no basis in reality. I'm sorry: Applets don't exist anymore, this obsession is outdated and futile. There is no rational justification behind breaking compatibility with almost every large Java project in existence to the extent that https://java.com still downloads Java 8.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Thursday January 18, 2024 @11:30AM (#39043795)
> No. JDK9/JDK11's adoption issues were due to --add-opens/--add-exports and the JDK developers hostile approach towards reflection.

No. JDK 9 and 11 kept all default accessibility the same as in JDK 8 (all you got was a warning) until it was changed in JDK 16: https://openjdk.org/jeps/396

The adoption issues were almost entirely due to non-portable libraries depending on internal implementation details that had changed in 9. Some were due to changes like that to the version string that dropped the "1." prefix for the first time.

> It's an obsession over "pure" code that has no basis in reality. I'm sorry: Applets don't exist anymore, this obsession is outdated and futile.

I don't understand the relationship to Applets, but the emphasis on integrity (a generalisation of safety) is all around the software world. It's the main reason for Rust's existence.

> to the extent that https://java.com still downloads Java 8.

That's because java.com is a website for end users downloading the JRE, which was discontinued after 8 (really in 11).

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by iscoelho on Thursday January 18, 2024 @12:56PM (#39045027)
> No. JDK 9 and 11 kept all default accessibility the same as in JDK 8 (all you got was a warning) until it was changed in JDK 16: https://openjdk.org/jeps/396

This is simply not true. This sounds like gaslighting in an attempt to rewrite history, which IMHO shows poor credibility from the JDK developers.

> I don't understand the relationship to Applets, but the emphasis on integrity (a generalisation of safety) is all around the software world. It's the main reason for Rust's existence.

Yes, and Rust has "unsafe". Java still has JNI. If programmers want to write unsafe code, they will write it, to the point of shipping a modified JVM. It's their computers. It's their code. It is a misguided and fruitless effort to push ideology on them.

> That's because java.com is a website for end users downloading the JRE, which was discontinued after 8 (really in 11).

I don't buy this.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Thursday January 18, 2024 @02:24PM (#39046370)
> This is simply not true. This sounds like gaslighting in an attempt to rewrite history, which IMHO shows poor credibility from the JDK developers.

I'm sorry, but default access was not changed until JDK 16. Here's the relevant section from JDK 9's JEP 261 (https://openjdk.org/jeps/261#Relaxed-strong-encapsulation):

In this release the strong encapsulation of some of the JDK's packages is relaxed by default... --illegal-access=permit opens each package in each module in the run-time image to code in all unnamed modules, i.e., to code on the class path, if that package existed in JDK 8. This enables both static access, i.e., by compiled bytecode, and deep reflective access, via the platform's various reflection APIs... This mode is the default in JDK 9.

> Yes, and Rust has "unsafe". Java still has JNI. If programmers want to write unsafe code, they will write it, to the point of shipping a modified JVM. It's their computers. It's their code. It is a misguided and fruitless effort to push ideology on them.

And Java will still gives you access to unsafe operations (even after this change) albeit with a flag that serves a similar purpose to Rust's unsafe, only done better. The goal is not to impose an ideology on programmers. If you read the JEP, the goal is for them, and the runtime, to know which invariants can be trusted and which can't (e.g. the JIT might want to optimise things based on the assumption that strings are final, but using unsafe code or native code makes that untrue). That's why JNI will also require a flag. As it is today, neither the runtime nor the application developer is able to know whether there's some code that could mess about with their assumptions.

> I don't buy this.

It literally says that on the front page of java.com: "Get Java for desktop applications." OTOH, the developer website, https://dev.java/, lets developers download JDKs only, as JREs no longer exist.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by krzyk on Wednesday January 17, 2024 @01:21PM (#39031406)
What is holding you back with upgrade to 17 or 21? Biggest pain was migrating 8 to 9, after that it is piece of cake.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by jmaker on Friday January 19, 2024 @04:51AM (#39053576)
The transitive hull of the deps
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by krzyk on Saturday January 20, 2024 @05:09AM (#39066106)
Yeah, but you usually want to have up to date deps either way, at least for security purposes.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by jmaker on Saturday January 20, 2024 @05:29AM (#39066193)
Of course, a classical trade-off. That’s why I’m taking time to discuss that here. The cost incurred by upstream updates reshuffling the hull makes it for hard budget and timeline decisions and risk management. That’s why larger projects with many deps prefer stable upstream, both the runtime and the libs. There’s also compliance and certification that come into play in regulated industries. Lots of moving parts to keep track of. I need to know what the upstream vendors are up to so we can plan in advance. With JDK, a big trouble is the push to JDK 17+ now within the ecosystem. Sonar Cloud for example now wants JDK 17+ starting next Monday. But you can’t even build your JDK 11 bound projects with Gradle that runs on JDK 17. You could separate the build stage from the run stage, of course, but that’s not really enough in the end. Same story with Spring. With .NET by contrast, the ecosystem still supports the legacy .NET Framework target and you can still run your antiquated projects. Not the same with Java anymore though. It’s pulling the rug.
Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Saturday January 27, 2024 @08:58AM (#39155568)
Most Java projects written for JDK 1.2 still run fine, unchanged and without recompilation, on JDK 22, provided that they don't make use of libraries that bypass the spec and reach for internals (or unless they're using discontinued technologies like Applets, although those would still be easy to port). Java's backward compatibility only applies to the spec, not to internal implementation details. Before internals were encapsulated in JDK 16, too many libraries reached for internals, and these non-portable libraries made their client applications non-portable without their knowledge. Those libraries reached for internals to add functionality and to improve performance, but in the process knowingly sacrificed portability but didn't make that tradeoff apparent to consumers.

Now that internals are encapsulated, upgrades are easier than they've ever been at any point in Java's history. Unfortunately, some large applications that depend on non-portable libraries — some of which were since abandoned — are stuck, but support for JDK 8 continues at lest until 2030.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by jmaker on Saturday January 27, 2024 @11:56AM (#39157225)
Thanks for these insights, Ron. Indeed, it’s mostly due to the dependencies that it’s not always simple to upgrade. Some Gradle plugins must also be updated. Code generators are rather fragile and some of them bluntly violate javac constraints but have been pulled as dependencies rather widely nonetheless. Yes, from the JDK vendor perspective on backwards compatibility and lifecycle support extensions, OpenJDK’s and Oracle’s track record is stellar, that's been my honest opinion throughout.

Beyond JDK though, the community has been trying to move forward in ways inconsistent with the backward compatibility for projects with dependencies such as those that you describe. This is what's causing most of the hesitation, exacerbated by the modality that there's little clarity on how to replace or upgrade such shenanigans.

From your comments, it turns out I’m quite unaware of the details of the transition to the encapsulation of the internals. I recall you mentioning that some time ago too. I’d appreciate a pointer to some overview, I feel like I’m going to need one before making a dive into the rabbit hole.

Re:JEP Draft: Deprecate memory-access methods in sun.misc.un... by pron on Saturday January 27, 2024 @02:15PM (#39158820)