Having a VM / JIT in the middle is a way to get there, but ultimately, different. At least in Java's case, my code targets an abstraction instead of hardware, so I don't get the full benefit of compilation. Performance is part of the story... access to the OS and hardware is another.

Even native compilers like clang will use LLVM, and LLVM IR can look a lot like JVM bytecode.

The VM abstraction is not a barrier to fully optimizing for your architecture, but rather how much time you can spend converting IR/bytecode into assembly, whether you do it at runtime, and whether runtime information lets you optimize even further.