There are two kinds of backwards compatibility at play: BC for compiled artifacts (let's call this Java ABI compatibility) and BC for language APIs (Java API compatibility). I have run into issues trying to compile an application with one JDK version and then run it on another (specifically, I believe JVMs will generally not run artifacts compiled with a later JDK version), but API backwards incompatibility is really rare in Java.
I do recall having to do a bit of cleanup when a previous project moved from Java 8 to Java 11, since a ton of deprecated APIs were pruned in Java 9, but those APIs were published by the JDK authors as standalone JARs, so I could remediate by adding a maven dependency. JVM backwards compatibility isn't perfect, but it's a much more stable target than other languages I've worked in.
https://www.azul.com/downloads/?version=java-17-lts&package=...
This is just how things are. It's Java fanfiction to say this isn't the case, and necessitates having multiple JVMs installed.