Because Java, particularly pre-8 Java, sucks so bad if you try to actually write your whole application in it.

(I always found this really frustrating when making the argument for Scala. Yes, Scala-the-language is more complex than Java-the-language. But it's not more complex than the pile of reflection magic that anyone who tries to write a nontrivial system in Java actually ends up using).

I try to do everything in plain-old Java, and my code just keeps running release to release.

From what I've seen: to add automagic functionality, syntactic sugar and wrapper methods for libraries. The JVM/Java language doesn't seem to have the tooling necessary to accomplish this natively, so it's the only reliable way to do it without relying on annotation processors having properly run (which still require some reflection magic, depending on the use case).

Reflection and class loader shenanigans to shoot them selves in the foot.

> Why do people bend over so far backwards doing reflection that isn't specified?

Either because there's no other way to do it, or because the official way is too slow.