News for hackers. Stuff that matters.
Re:Faster TypedArrays: Vector Addition in WebAssembly by eyelidlessness on Sunday January 02, 2022 @09:16PM (#29776563)
> JIT warmup

No kidding, this can be seriously misleading.

IMO JS (and likely many other JIT VM) benchmarks should run:

- multiple cold starts

- multiple pre-warmed starts (essentially run your bench twice, discard the results, run it again)

- isolated cases

- shared runtime cases with evenly distributed ordering

All of those will reveal different performance characteristics you may want to know about, and people evaluating your results should be able to know about. Cold starts are a worst case scenario of each implementation, warmed should show a more ideal outcome. Isolated will again highlight the perf of each implementation on its own merits, whereas shared runtime should help highlight how interacting with other real-world code affects VM optimization of that implementation.

Re:Faster TypedArrays: Vector Addition in WebAssembly by xpl on Sunday January 02, 2022 @11:35PM (#29777376)
Yeah, properly benchmarking JS code is hard because of how complex is the V8's optimizing compiler infrastructure. I once made an online tool for measuring run times of JS code, with [hopefully, lol] proper warmup and protection against possible dead code elimination optimization.

There are a few clickable benchmark examples in the readme: https://github.com/xpl/what-code-is-faster#what-code-is-fast...

Re:Faster TypedArrays: Vector Addition in WebAssembly by ignoramous on Monday January 03, 2022 @02:52AM (#29778374)
> No kidding, this can be seriously misleading.

Reminds me of prof. Emery Berger's talk, Performance Matters: https://www.youtube-nocookie.com/embed/r-TLSBdHe1A