> There is, however, a problem with build tools not supporting newer JDK features

This is important point.

From my experience in Enterprise IT the build pipeline is roughly at JDK-8 level in terms of generating deployable artifacts. I have contorted by maven build config to invoke jlink on builder nodes to make standalone artifacts that won't need JDK on target machines. But it is not optimal for few reasons 1) I have to manually track what JDK modules I need to include in build. 2) Its hard to get new JDK installed on build machines for ones, forget the dot upgrades to keep it latest bug fix level. Their reasoning is build machine is just to invoke maven build. Only target machines need latest patched JDKs. 3) From JDK-8 they are now directly jumping to make docker/kubernetetes builds which are fine but totally different direction if I am just looking to make no-fuss standalone Java application to run on a Linux VM.

As one can see some of these are organizational issues not necessarily technical one. But as single developer with no power to force IT infra teams on supporting alternate workflows I am basically stuck with JDK-8 workflows.

What is missing in setup is somehow to tell maven I want to generate a standalone Java app using jlink for linux ver.N target so gather up all modules / java binary and other tools required based on information in this pom file without relying on JDK version of builder . This should be able to run builder machine so it remains integrated with current build pipleline.

P.S.

Oracle's Java Platform Group (JPG), the group behind OpenJDK, had nothing to do with Jipher, but the number of people who contributed to OpenSSL in the past year alone is roughly the same number as the people working on Project Loom, and rewriting OpenSSL from scratch would have obviously required more people than those required for OpenSSL maintenance, so it would have been a project roughly the size of Loom. If it were to actually do the job, it would also require a commitment to long-term maintenance (something that's often missing from various rewrite attempts). I don't think it's necessarily the best use of resources, at least by us, and I don't think it's sad at all that we're investing our resources in things with better bang-for-the-buck. I think that projects like Loom and Panama and Leyden have a much better ROI than an OpenSSL rewrite.

Also, most native libraries work perfectly fine on virtual threads and with no degradation in performance. The only thing to watch out for is frequent blocking in native code.

But Java already has a cryptography stack (including TLS) in pure Java shipped as part of the JDK which has been maintained for years. So manpower isn't the issue, the problem is that Java code can't be easily used by anything else. The awkwardness of JNI cuts in both directions.

It could be solved if there was a way to generate a shared library from Java code with an auto-generated reasonable-ish C API. Then people could take the Java SSL code (+bouncy castle or whatever) and implement the OpenSSL API. Ditto for lots of other libs. But as no such thing exists, everyone gathers around the C impl because that way you get the most stakeholders.

> The awkwardness of JNI cuts in both directions.

On the consumer side FFM solves that; not awkward anymore! On the host side, well, even Go libraries (and I mention Go because I think Go is considered by some as being lower level than Java for reasons passing understanding) aren't easily shareable by other languages.

> It could be solved if there was a way to generate a shared library from Java code with an auto-generated reasonable-ish C API.

But why is that a problem worth spending our time on considering that C programmers probably won't appreciate hosting a JVM for that, and that Java is more popular than C as it is? No language is trying to control everything these days. The anomalous and short era of single language dominance is over, at least for a while. Java isn't the one dominant language, but neither is any other. Spending effort to go after a relatively small market share seems like a miscalculation. The market of popular open-source C libraries is both very important and minuscule. It would be like saying that iOS should focus its resources on going after the home-brew Raspberry Pi market.

> But as no such thing exists, everyone gathers around the C impl because that way you get the most stakeholders.

Why is that a problem, though? Libraries that are shared across languages and runtimes are very important, but their number is also very small. Big and foundational are two very different things in software, and we're in an era of specialisation, not consolidation. Don't confuse "a problem" with "a problem worth solving".

>> How do you ship a Java library that uses native code? Python/Node/Ruby worlds know, but OpenJDK ignores the question.

>It doesn't. jmod files and the jmod tool were added in JDK 9 precisely for that (and other things, too).

I have never seen a lib package native dependencies using jmod. It's not even clear to me how this would be done. Everyone I'm familiar with bundles binaries inside the jar. Can you point to an example? I would love to learn a better way of doing this.

That's what JavaFX does (https://openjfx.io/openjfx-docs/#modular), but the problem is that Maven and Gradle don't support that well, which is why few libraries do that.

The idea is that an application is best deployed with a custom runtime created with jlink (well, all Java runtime these days are created by jlink and so every Java program uses jlink whether it knows it or not, but few use it directly because, well, Maven and Gradle don't support jlink well, and so few applications use jlink well), and jmod files are jlink's input from which it generates the image.

Anyway, build tools aside, a library can be distributed as a jmod file, consuming it with jlink places the native libraries in the right place, and that's about it.

JavaFX doesn't use jmods, not really. Try following the tutorials for getting started and see for yourself: you end up using custom JavaFX build system plugins that download jar versions of the modules, not jmods. Also some alternative JDK vendors pre-ship JavaFX in their JDK spins to avoid people having to use jlink.

I've worked with jlink extensively. The problems are greater than just Maven and Gradle, which at any rate both have quite sophisticated plugins for working with it. There was just a major error at the requirements analysis phase in how this part of Java works. Problems:

1. You can't put jmods on the module path, even though it would be easily implemented. They can only be used as inputs to jlink. But this is incompatible with how the JVM world works (not just Maven and Gradle), as it is expected to be able to download libraries and place them on the class/module path in combination with a generic runtime in order to use them. Changing that would mean every project to get its own sub-JDK created at build time, which would explode build times and disk space usage (jlink is very slow). So nobody does it, and no build system supports it.

2. When native code gets involved they are platform specific, even though (again) this could have been easily avoided. Even if JVMs supported jmods just like jars, no build system properly supports non-portable jars because the whole point of a jar is that it's portable. It has to be hacked around with custom plugins, which is a very high effort and non-scalable approach.

3. jlink doesn't understand the classpath at all. It thinks every jar/jmod is a module, but in reality very few libraries are usable on the module path (some look usable until you try it and discover their module metadata is broken). So you can't actually produce a standalone app directory with a bundled JVM using jlink because every real app uses non-modularized JARs. You end up needing custom launchers and scripts that try to work out what modules are needed using jlink.

4. The goal of the module system was to increase reliability, but it actually made it lower because there are some common cases where the module system doesn't detect that some theoretically optional modules are actually needed, even in the case of a fully modularized app. For example apps that try to use jlink directly are prone to experiencing randomly broken HTTP connections and (on desktop) mysterious accessibility failures, caused by critical features being optional and loaded dynamically, so they have to be forced into the image using special jlink flags. This is a manhole sized trap to fall into and isn't documented anywhere, there are no warnings printed. You are just expected to ship broken apps, find out the hard way what went wrong and then fix it by hand.

At some point the JDK developers need to stop pointing the finger at build tool providers when it comes to Jigsaw adoption. It's not like Maven and Gradle are uniquely deviant. There are other build systems used in the JVM world and not one of them works the way the OpenJDK devs would like. They could contribute patches to change these things upstream, or develop their own build system, but have never done it so we end up with a world where there are ways to distribute libraries that work fine if you pretend people never moved beyond 'make' and checking jars into version control.

> You can't put jmods on the module path, even though it would be easily implemented.

Yes, it could be easily implemented, although things would work nicely even without that. You make it sound as if there would have been proper build tool support if that were the case, but JARs can be easily put on the module path and still build tools don't properly support even that yet.

> When native code gets involved they are platform specific

Well, yeah. Native code is platform specific, and that's one of its main downsides and why most libraries don't (and shouldn't) use it. But when it is used, it's used for its upsides.

> At some point the JDK developers need to stop pointing the finger at build tool providers

All of the problems you mentioned could only be solved by build tools, but we're not pointing fingers in the sense that we blame build tools for things being bad. After all, modules have been very successful at allowing us to do things like virtual threads and FFM and remove things like SecurityManager. Build tools providers can have their own priorities, just as we do. But if you want to enjoy your own modules or 3rd party modules then you'll need good support by build tools.

> They could contribute patches to change these things upstream, or develop their own build system, but have never done it

Yet. There were more urgent things to work on, but maybe not for long.

Gradle will put JARs on the module path if the app itself is a module. If JMODs were an extension to the JAR format rather than a new thing, and if there was no such thing as the module path (modules were determined by the presence of their descriptor), then nothing new would be needed in build systems and everything would just work.

No, working with modules requires the ability to place some JARs on the classpath and some on the classpath arbitrarily. Any kind of automatic decision regarding what should be placed on the classpath and what on the module path misunderstands how modules are supposed to work. The -cp and -p flags don't and aren't supposed to differentiate between different kinds of JAR contents. If they did, you're absolutely right that the different flags wouldn't be needed -- the JDK could have looked inside the JAR and automatically say this is supposed to be a module or not; the reason they are needed is because that's not what the flags mean.

A module in Java is really a different mode of resolving classes and applying various encapsulation protections. If x.jar is a JAR file, `-cp x.jar` means "apply certain rules to the resolution and access protection of the classes in the JAR" while `-p x.jar` means "apply different rules for those same classes". In most situations both `-cp x.jar` and `-p x.jar` would work for the same x.jar, applying those different rules, regardless of whether x.jar declares module-info or not. The decision of which rules need to be applied belongs to the application, not the JAR itself; being a module or not is not something intrinsic to the JAR, it's a rule chosen by the application.

It's a little like choosing whether or not to run a program in a container. You can't look at the executable and say this should run in a container or not. The decision of whether to set up a container is up to the user when configuring their setup. module-info basically means: if the user sets up a container to run me, then these are hints to help configure the dockerfile. In an even stronger sense, a module is similar to a class loader configuration; some classes may work better with some classloader configurations than with others, but ultimately the decision on the setup is not something intrinsic to the classes but to the application that loads them, and the same goes for modules.

So having the build tool or the JDK guess which rules need to apply to which classes makes as much sense as having them guess the proper heap and GC configuration the application needs -- you can have an okayish default, but for serious work the app developer needs to say what they want. The JDK makes it very easy; build tools make it very hard.