this has been the case in the past, but Java is gritting its teeth and making some breaking changes in the JDK11/16/18 era.
libraries that mess with the unsafe/internal namespaces are the most prominent example - there are a ton of legacy libraries that are going to break from that change unless updated - but there's also a big push for modules and for classes to encapsulate themselves more strongly.
Then why do I have to have multiple JVMs installed, which I need to go through via trial-and-error, if I'm given some random java application?
You don't. Almost any Java program written ever will run on Java 18.
This is just how things are. It's Java fanfiction to say this isn't the case, and necessitates having multiple JVMs installed.
There are two kinds of backwards compatibility at play: BC for compiled artifacts (let's call this Java ABI compatibility) and BC for language APIs (Java API compatibility). I have run into issues trying to compile an application with one JDK version and then run it on another (specifically, I believe JVMs will generally not run artifacts compiled with a later JDK version), but API backwards incompatibility is really rare in Java.
I do recall having to do a bit of cleanup when a previous project moved from Java 8 to Java 11, since a ton of deprecated APIs were pruned in Java 9, but those APIs were published by the JDK authors as standalone JARs, so I could remediate by adding a maven dependency. JVM backwards compatibility isn't perfect, but it's a much more stable target than other languages I've worked in.
https://www.azul.com/downloads/?version=java-17-lts&package=...
There are occasionally breaking changes but most old jars out there will run fine in the latest runtime, so the answer to your question is going to depend on how squirrely your use case is.
JVM performance is generally fine, and the language prevents you from introducing whole classes of memory safety bugs. You can still shoot yourself in the foot with native (JNI) extensions, but the decent performance of the JVM makes developers use JNI far less frequently than, say, Python or Ruby devs reach for compiled libraries.
JVM byte code decompilation is fairly trivial, so tools for the security scanning of artifacts are cheaper and more fully featured than similar tools for scanning machine code.