You could still use maven w/o an IDE, it's really not that bad.

It's mainly for compiling automatically when the code changes and for hot reloading. It's important for working with web apps. For desktop maybe not but it's been maybe 10 years since I built a desktop Java application.

Maven is actually very important since it is a "generic" project format that all IDEs understand, it handles dependency management/downloading, etc. Ant and Gradle are nowhere near as useful since they are not a format an IDE can understand natively. They are a build script, not a project file, so you need to keep another project file in sync for each client you're using.

You don't even necessarily need to be building a Java target for Maven to be useful. It's also a common way to build continuous-integration build scripts for things like Docker packages, database artifacts, etc.