But, generally, I'm not interested in whether "for" loops run faster in language X or language Y. I am interested in whether some languages encourage code styles that happen to be easily optimized (I'm also interested in whether some languages encourage code styles that are easily maintained, easily extensible, or easily run into a ditch).
This paper is meaningful to somebody, but it's clearly written by people who spend most of their thinking about things other than programming. That's not an insult; it's a simple fact that a lot of programs are written by people who are experts in other fields. I happen to work with a former physicist who has experience writing programs to run on one of the nation's largest supercomputers. The depressing thing (to me) is that while he knows how to write numerical programs that run close to a computer's theoretical maximum speed, that information is only well-known among a small group of people (namely people who have to show that if they're allowed to use a supercomputer, they won't make it run wasteful code). I wish it were shared more broadly.
(Well, unless I am just wrong on the feats that JIT'ed code can perform nowdays.)
Compare to C++ where closures can be used with virtually no penalty and aggressively optimized. Even C with its trick of passing an extra *void for environment comes out shining performance-wise in comparison.
Edit: There's a good discussion of the problems that JVMs have in this area at http://www.azulsystems.com/blog/cliff/2011-04-04-fixing-the-....
"The Problem is a hard one to solve; (i.e. it’s not the case that we’ll wave our wands and do a quick fix & rebuild HotSpot and the problem will go away). John Rose, Doug Lea and I all agree that it’s a Major Problem facing JVMs with long and far reaching implications."
Maybe progress will be made soon though, with more pressure caused by increasing use of closures in the new streams library and with anonymous function syntax available now.
JVM inlining is unmatched by static compilers now and is going to only improve once techniques described in the Azul paper are used. Sure, JVM has a ton of other problems compared to C++, but inlining is not one of them.
BTW: I agree with the point that JVM optimizations are not enough and you often have to know what you're doing and optimize by hand appropriately.
Now, can this be improved? Of course. Scala, for example, does a good job of not requiring primitive arrays. This is done with compile time tricks, though, not runtime (JIT) ones.
No way. It may be native C, but it's not native C you've hand-coded yourself.
They also executed their test code on poorly performing operating systems not designed to execute high performance code, ie. they ran their tests on consumer desktop general purpose operating systems instead of a more specialized system designed for the task... or at the very least a more specialized OS such as a flavor of BSD or Linux without the desktop and other generalized packages.
I don't really consider this a well done test, and the results are only surface-level... certainly nothing you should base a serious decision on.