Exactly. The hard part of Java is not the language per se, but the myriad of IDEs and XML configuration files.
To be honest, as a Java veteran, there is very little XML any more apart from a bit of Tomcat config usually. It's all gone in favor of annotations which are much better.
It ain't J2EE 1.4 days any more (thank fuck).
The only thing I've done involving XML recently was setting tomcat 7 to serve a war file from the root site and that took about 2 minutes to work out.
I haven't used an XML config file in literally years. Either use embedded Jetty directly or use a nice thin wrapper like DropWizard.
The language has not proved to be too disturbing, happily, but I'm finding the tooling and ecosystem to be rather arcane and difficult to get a grip on at first. Build tools, a choice of JVMs and JDKs, "enterprise" this or that.. it seems all the stuff around the language is the real minefield than the relatively simplistic language itself.