The further you move away from the Java typesystem or execution model, the less fun your life will be and the fewer benefits you'll get from the JVM: moving to dynamic typing will cost you a large amount of performance, for example, and you'll have to fight the Java type system a fair amount. I can only imagine what it would mean to do lazy-evaluation in a JVM language.
If it matches what you're trying to do, though, it's really a great platform: the bytecode format is pretty simple and easy to learn, the JVM runs of pretty much any platform you need, and it does a huge amount of optimization for you. If you're doing a statically-typed language and compiling down to Java-like bytecode (as opposed to handling dispatch yourself, as you have to do in a dynamically-typed language), you also basically get debuggers and profilers for free.
It's definitely a little weird, though, that they'd set up the motivation for HLVM as due to the weaknesses of the CLR's cross-platform capabilities, without mentioning the JVM at all, given that the JVM was, in spirit at least, the precursor to the CLR and still the most widely-used higher-level VM and one that works basically everywhere.
Isn't the DaVinci/MLVM project an example of how Sun actually tried to move towards dynamic and non-java languages in the JVM? invokedynamic & method handles are already in the upcoming java 7, and that goes a long way towards supporting dynamic languages on the JVM.