> faster even than hand-tuned assembly

I think you and I are working off different definitions of what that means, as my definition doesn't really allow for a faster implementation (unless there's some really spooky stuff going on in the compiler). I suspect you mean faster than a popular hand tuned implementation.

Obviously Zig's output (courtesy of its LLVM backend) can be expressed and distributed as assembly code, but it's not hand-tuned, it's compiler-generated.

Whether the hand-tuned assembly code they used was the fastest out there, I'm not sure. I'd hope so, or they're being misleading.

I might get time to re-watch the relevant part of the video - it's around the 21 minutes mark - https://youtu.be/Z4oYSByyRak?t=1292

My point is that all the fastest hand-tuned assembly has to do to match the compiled output is to do the same thing. The compiler has no ability to do anything a hand-tuned assembly cannot, by definition of how this all works.

So, saying it's faster than hand-tuned assembly doesn't really make sense, but saying it's faster than current optimal hand-tuned assembly could, but that also probably requires searching for a not-very-well tuned problem, as any optimization the compiler could apply is something that could be applied by hand in the hand-tuned assembly (even if it requires a lot of work to do so).

A general statement that might be made (and would also be very impressive) would be that it compiles to implementations that are on par with the current best hand-tuned assembly versions.

The small edge case where the original statement might be true is where some AI/ML/evolutionary techniques are applied and the assembly is faster in the generated output but we don't know why. That is, the speedups are not because of rules and heuristics, but derived by some complex process which we have little insight into, and thus can't manually replicate (which is what I meant by spooky stuff).

I mean, there's nothing a computer does playing chess that a human couldn't, and yet the world's best players simply can't beat the world's best chess-playing programs.

This assumes that the problem of a chess game and providing optimal solutions for an algorithm defined in a higher level language (or pseudocode) are similar enough that the same strengths apply. They may be, but I don't think that's something you can assume without some sort of evidence.

Though, I guess the same could be said of my prior statements. To my knowledge, compilers aren't generally looking through a search space for optimized solutions as many common game AI algorithms do (which would would be in line with my statements about AI/ML/etc above). I don't have the relevant citations or experience to state that with authority though.

Andrew explains in the video that the speedup is mostly due to compile-time speed ups and potentially the use of the rorx instruction. It cannot be faster than the fastest assembly because the by definition is the fastest assembly.

> compile-time speed ups and potentially the use of the rorx instruction

Indeed, but the 'compile-time speedups' are a legitimate point in favour of the compiler. If they didn't occur to the assembly programmer, or struck them as too complex to pull off, then the compiler deserves the point.

Have to say I don't follow why the hand-tuned assembly doesn't use the rorx instruction. It's not mentioned in the assembly file, at least, but I thought that was the point? https://www.nayuki.io/page/fast-sha2-hashes-in-x86-assembly

Also, it's neat to see instruction-selection being so significant. Generally might expect cache/branch behaviour to be the kicker.

> It cannot be faster than the fastest assembly

Well, 'fastest assembly' is the domain of so-called 'superoptimisers', and has us pushing at the stubborn bounds of computability.

We were talking about hand-written assembly-code, compared to compiler-generated. Odds are that none of the binaries tested were the optimal assembly.

The only interesting question is whether the hand-tuned assembly code they tested, was the fastest available at the time. If not, the whole demonstration is a straw-man, of course.

Also I don't like that the winning Zig program runs for so much longer. A good benchmark should provide ironclad assurances that no candidate is getting an unfair advantage re. loading/'warm-up'.

I don't believe it is possible to hand tune every program to beat a compiler.

More a question of practicality, no?

Programmers are creative, compilers are well-tuned machines. Sometimes it's tough to beat a compiler's code, but I'm not sure that 'impossible' is meaningful.

Monkeys with typewriters could function as a highly parallel superoptimiser, after all.

When you realize the compiler's optimizations only account for about 10% of the total program's performance you find that the other 90% is entirely up to the programmer.

Architecture, data structures, batching operations, memory locality, and a bunch of other metrics are all concepts the compiler can't really help you with whatsoever and they have a much larger impact on performance than the 10% the compiler is actually able to optimize.

The problem is that either programmers don't care, or they can't make the distinction between premature optimizations and architecture planning.

You're right to emphasise good data-structures and algorithms (also concurrency, parallelism, etc), but compiler optimisation is nothing to sneeze at. '10%' is laughably off-base.

From a quick google: compiler optimisation can accelerate CPU-bound code to over 5x the unoptimised performance. https://www.phoronix.com/scan.php?page=article&item=clang-gc...

They seem to be benchmarking very specific things and not actual applications. These numbers do not hold in the real world.

In a sense you're right, but hand-tuning assembly is kind of an orthogonal problem to determining whether you're using the right algorithms and data structures.

Where do you get that 90/10 split from? Just curious.

Talks from Mike Acton and Scott Meyers, specifically "Data-Driven Development" and "CPU Caches and why you should care" respectively.

I forgot exactly where I got that number, but it's been a pretty good metric so far.

In a nutshell; the compiler is great a micro-optimizations and absolutely terrible at macro-optimizations. The former will get you a few percent of perf boosts while the later usually results in orders of magnitudes of performance gains.

Its near impossible to apply macro-optimizations at the end of a project without massive refactors.

Cool, I'll check those out, thank you!

I believe the 10/90 number may be from this talk, though I don't have time to rewatch it to confirm: https://www.youtube.com/watch?v=rX0ItVEVjHc

But it should be possible to hand tune to match a compiler, right? And that would mean the compiler was not faster, which is the general claim I was calling into question.

How so? Compilers aren't magic, you can - by definition (Assuming the current state machine style (no AI or nuthin') of compilers ) - you can just manually compile it.

Compilers are clever, but humans can also be really clever: They know how to think - You can change the algorithm entirely.

For practical reasons however (Premature optimisation also), trust your compiler.

It should be noted that 0.2.0 is significantly different than master in regards to pointer syntax due to pointer reform [0]. There are a lot of other changes too, but that one's the most noticeable.

[0] https://github.com/ziglang/zig/issues/770

Does Zig have a memory management story other than C's? Anything like Rust's borrow checker?