I quite disagree. Look at JVM performance, there’s been huge improvements over time and a Java program compiled 20 years ago will get the same improvements as one compiled today.

Not only that but you can expect to get better performance from the JIT than AOT because the JIT can perform optimisations which are impossible for the compiler.

The compiler can also perform optimizations that are impossible for a JIT because they're too expensive. JITs have to balance execution speed with compilation time. It's not so clear cut which approach in theoretically superior. I also don't know any languages where a JIT and an AOT compiler have gotten equivalent amounts of engineering hours so that we could have a fair comparison.

Graal can run in dynamic and aot compilation mode. If you feed profiles into aot from a test run you can get pretty close to dynamic compilation peak performance.

And if you don't the gap is about 20% iirc.

> you can expect to get better performance from the JIT than AOT

And you are likely to be disappointed. I think these claims have been made for the last 20 years, and Graal might be the first to deliver something general - all previous successes were very limited to small niches.

I like JITs and what they offer, but very consistently for me and almost everyone I know, in practice AOT works better (faster, consistent, no "heuristic changed in JIT version and now it's slow" surprises).

> Not only that but you can expect to get better performance from the JIT than AOT because the JIT can perform optimisations which are impossible for the compiler.

I can’t help myself, but the JVM performs at par with managed AOT languages (Go, Haskell, OCaml, etc) and strictly slower than the unmanaged AOT languages (C, C++, Rust). The JVM does outperform various AOT Java compilers, but that’s probably an artifact of the decades of optimizations that went into the JVM; the same investment in an AOT compiler would close the gap to within the margin of error. Anyway, sorry for my compulsive nitpicking; hopefully it was interesting.

Lots of people believe this but it's false.

JVM vs C++ depends very much on the code shapes and what the code is doing. C++ that's very heavy on virtual function calls can be faster written as Java. On the other hand if you use a lot of SIMD intrinsics and things like that, C++ can be a lot faster.

W.R.T. AOT vs JIT, as others are pointing out, Graal is a compiler that can execute in both modes, i.e. it's a comparable compiler. JITC is about a 20% win for relatively static languages like Java and can be massively larger (like 4000%+) for other languages like JavaScript or Ruby. In fact the nature of very dynamic languages like Ruby or Python or JavaScript mean there's little point trying to AOT compile them at all because there'd be almost no optimisations you could safely do ahead of time.

You make it sound like Java is neck-and-neck with C++ in general. There are definitely cases where naive Java is faster than naive C++, but those cases are infrequent. Java is generally in the ballpark of 1/2 C++ speed alongside Go and C# in the general case.

> W.R.T. AOT vs JIT, as others are pointing out, Graal is a compiler that can execute in both modes, i.e. it's a comparable compiler. JITC is about a 20% win for relatively static languages like Java and can be massively larger (like 4000%+) for other languages like JavaScript or Ruby. In fact the nature of very dynamic languages like Ruby or Python or JavaScript mean there's little point trying to AOT compile them at all because there'd be almost no optimisations you could safely do ahead of time.

It seems like you mistook my point for "AOT makes things fast and should be used everywhere" or "JIT is fundamentally slower than AOT"; my only point was that the JVM specifically isn't faster than AOT languages nor is it faster than AOT Java because of JIT specifically (which was how I interpreted jahewson's comment above). It sounds like I agree with you--Graal seems really promising and JIT is pretty great, especially for dynamic languages. :)

Well, C++ is such a flexible language that it's hard to say what general C++ code looks like. I'd say I'd expect Java to either match or even beat C++ for general "business code" which is pretty vague but is typified by lots of layers of abstraction, lots of data transformation, hash maps, string manipulation etc. Modern compilers like Graal are very good at analyzing and removing abstraction overhead. I'd expect C++ to stomp Java for any sort of numerics work, media codecs, 3D engines, things like that.

my only point was that the JVM specifically isn't faster than AOT languages nor is it faster than AOT Java because of JIT specifically

Hmm, but is there such a thing as an AOT language? You can interpret or JIT compile C and you can interpret, JIT or AOT compile Java too.

I think it's clearly the case that JITC Java is faster than AOT Java. It was maybe unclear for a long time but Graal and SubstrateVM let us compare now. You pay a big performance hit to use a VM-less AOT Java compile. Java is sort of a half-way house between a really dynamic language and a really static language.... it's got dynamic aspects and also static aspects.

> Hmm, but is there such a thing as an AOT language? You can interpret or JIT compile C and you can interpret, JIT or AOT compile Java too.

You're right that my language was imprecise, but my point stands. JVM does't make Java faster than popular AOT implementations of (for example) Rust or C++ or C, and JVM is almost certainly faster than AOT Java implementations because of decades of optimizations, not because JIT is inherently better than AOT.

> I think it's clearly the case that JITC Java is faster than AOT Java. It was maybe unclear for a long time but Graal and SubstrateVM let us compare now.

Graal is shaping up to be a real game changer, but it's unproven and it's exceptional among JIT implementations. If Graal is all that it's cracked up to be, I wouldn't be surprised if there is a time when all popular JIT implementations perform like Graal, but until such a time, JIT alone doesn't have any clear performance advantages over AOT in general.

> because the JIT can perform optimisations which are impossible for the compiler

Is there an example of at least one such optimization? As JIT is not magic, it also compiles things ahead of time, but a little bit ahead of time and it is very constrained on how much time and resources it can waste. It also can't afford to slow down interpreter too much, can't do excessive profiling, can't do big changes to the code that touch half of all the functions and data structures for example, etc. While AOT can do that and I can't think of anything JIT does that AOT can't do.

One such example is dynamic re-compilation in Java, with e.g. inlining or uninlining based on recent profiling. This would handle the case where a certain (large-ish) function is on a branch that's often taken for some stable amount of time (where inlining gets rid of method call overhead), then often not taken for other periods of time (where un-inlining gets rid of the instruction cache bloat).

Artificial example:

    while (true) {
        if (timeOfDay < NOON) {
            computePi(); //should be inlined before noon, function call after noon
        } else {
            computeE(); //inline after noon, function call before
        }
    }

@JIT vs. AOT: i used to think that, too. but graal came out and got 90% of the JIT speed right from the start with a fraction of the time hotspot had for optimization (and of course, with better startup times).

Yes, it seems Graal might finally deliver on the "JIT is better because it is better informed" promise after 20 years. I'm holding my fingers crossed, but I think we need a little more experience with it before victory can be declared.

uh, a misunderstanding, my mistake. what i meant was that graals native-image AOT compiler with the substrate VM achieves 90% of hotspot JIT execution speed pretty much since the project was launched. you may be right that graals JIT is still faster, i don't know about that.

What about luajit?

luajit is a great triumph for dynamic compilations and jits; and the fact that it compares favorably to C code (about 70-85% in some of my experiments) is utterly amazing. Especially given that it's essentially the work of one person.

And yet, regardless of how much typing info you give it, it loses out to good AOT compilers, for a similarly implemented algorithm.

GraalVM is unique in that it seems to do better than AOT in some cases. I'd wait a couple more years before declaring JIT victory, though.

The belief is that JITs have information, such as variable values and memory access patterns, that an AOT does not. Well, so far hardly any JIT uses that info, and AOTs do have them from through profile feedback; And also, JITs have to amortize compilation effort and runtime saving, whereas AOT doesn't. So the "JIT is obviously >= AOT" is not foregone conclusion.

I very much agree with you!

> You only really get any improvement in runtime if your compiler guarantees anything (and bakes it into whatever it is compiling).

I meant this in reference to runtime speed optimisations coming from types. V8 is improving every day!

Wouldn't it be good if JavaScript also starts supporting python like type annotations (completely optional, but unlike python used to guide the optimizer. Given that some non trivial amount of JS is already generated after type checking (typescript, flow, and all the statically typed languages compiling to JS), all this compile time type info could be made available to the optimizer.

For speed purposes? This post is for example already talking about how fast JS is getting without types.

I'd say it would be a lot easier to simply focus on the WebAssembly format, you can write your code in a number of (typed) languages and compile into wasm which is designed to be fast.

Javascript already comes with a few decades of bagage, and things are already changing really fast (ES6 and newer versions). Not everyone likes types and the people who do can use typescript (or flow). And the people who need insane performance (games, etc) can use WebAssembly.

Yes for speed purposes. And yes some people may not like types, that's why I said completely optional. Anyways the optimizer tries to figure out the types on it's own, so give it a non blank slate to start with. You are correct in noting that js is already fast enough, but the problem is the sort of performance gotchas(order of keys in objects) as mentioned in the post aren't really obvious during development time. Types may help level the unpredictability.

Python totally ignores them at runtime, PHP has this but it actually slows down the code a bit. It does make catching bugs easier.

That's basically what asm.js was, just less friendly to human readers.

asm.js put a type information to every single expression node, something you would never do even in the most statically typed languages.