The further you move away from the Java typesystem or execution model, the less fun your life will be and the fewer benefits you'll get from the JVM: moving to dynamic typing will cost you a large amount of performance, for example, and you'll have to fight the Java type system a fair amount. I can only imagine what it would mean to do lazy-evaluation in a JVM language.
If it matches what you're trying to do, though, it's really a great platform: the bytecode format is pretty simple and easy to learn, the JVM runs of pretty much any platform you need, and it does a huge amount of optimization for you. If you're doing a statically-typed language and compiling down to Java-like bytecode (as opposed to handling dispatch yourself, as you have to do in a dynamically-typed language), you also basically get debuggers and profilers for free.
It's definitely a little weird, though, that they'd set up the motivation for HLVM as due to the weaknesses of the CLR's cross-platform capabilities, without mentioning the JVM at all, given that the JVM was, in spirit at least, the precursor to the CLR and still the most widely-used higher-level VM and one that works basically everywhere.
Isn't the DaVinci/MLVM project an example of how Sun actually tried to move towards dynamic and non-java languages in the JVM? invokedynamic & method handles are already in the upcoming java 7, and that goes a long way towards supporting dynamic languages on the JVM.
I know that I could add options for .jar files to automatically invoke the JVM, but then not all use a Main class, so that's not a guaranteed fix.
Maybe with the new generation of "cool" languages for the JVM we may start to see some change on this front. Oracle may have some interesting ideas as well, as new "owners" of the Java platform...