Re:Faster TypedArrays: Vector Addition in WebAssembly
by brrrrrm on Sunday January 02, 2022 @03:23PM
(#29773385)
It's definitely a bit faster with the unroll, but on my machine it's not by too much. I've added that idea to the interactive benchmark if you'd like to check it out!
Re:Faster TypedArrays: Vector Addition in WebAssembly
by eyelidlessness on Sunday January 02, 2022 @09:16PM
(#29776563)
> JIT warmup
No kidding, this can be seriously misleading.
IMO JS (and likely many other JIT VM) benchmarks should run:
- multiple cold starts
- multiple pre-warmed starts (essentially run your bench twice, discard the results, run it again)
- isolated cases
- shared runtime cases with evenly distributed ordering
All of those will reveal different performance characteristics you may want to know about, and people evaluating your results should be able to know about. Cold starts are a worst case scenario of each implementation, warmed should show a more ideal outcome. Isolated will again highlight the perf of each implementation on its own merits, whereas shared runtime should help highlight how interacting with other real-world code affects VM optimization of that implementation.
Re:Faster TypedArrays: Vector Addition in WebAssembly
by xpl on Sunday January 02, 2022 @11:35PM
(#29777376)
Yeah, properly benchmarking JS code is hard because of how complex is the V8's optimizing compiler infrastructure. I once made an online tool for measuring run times of JS code, with [hopefully, lol] proper warmup and protection against possible dead code elimination optimization.
Re:Faster TypedArrays: Vector Addition in WebAssembly
by IvanK_net on Monday January 03, 2022 @04:38AM
(#29778954)
If a 50-character piece of JS is running for two seconds, it was definitely optimized at some point. And even a most advanced optimization of 50 characters would not take more than 0.1 seconds.
Even with a warmup, the unwrapped version is more than 3x faster.
Re:Faster TypedArrays: Vector Addition in WebAssembly
by IvanK_net on Monday January 03, 2022 @02:04PM
(#29784829)
BTW. I replaced every "let" and "contst" with "var" in your code, and it seems to run about 15% faster: https://jsfiddle.net/05xd1oft/
Inserting the loop into a function really speeds it up in Chrome. Also, I have noticed, that for Float32Array, the rolled / unrolled performance is almost identical, but for Uint32Array, unrolled is about 30% faster: https://jsfiddle.net/aq06p94f/
Re:Faster TypedArrays: Vector Addition in WebAssembly
by Matheus28 on Sunday January 02, 2022 @01:08PM
(#29771589)
If we're not counting the time to zero out the array, it seems that typed arrays are slower than plain javascript because it's converting to & from floats/doubles back and forth. Try the same with Float64Array. My microbenchmark says f32 is 15% slower than f64 on chrome: https://jsbench.me/gnkxxkjag8/1
Re:Faster TypedArrays: Vector Addition in WebAssembly
by brrrrrm on Sunday January 02, 2022 @01:31PM
(#29771900)
Great catch! I assumed the JIT would identify f32 arithmetic but I guess that isn't really valid numerically. I wonder if there's a way to use Math.fround[1] on your benchmark to get expected speedups?
Re:Faster TypedArrays: Vector Addition in WebAssembly
by olliej on Sunday January 02, 2022 @02:46PM
(#29772904)
Yup, the spec requires computation to be as doubles and so any computation must be done that way as the end result is observable.
I didn’t know about fround but I suspect its primary use case is for trying to catch floating point overflow during double arithmetic, because again precision difference is observable.
Re:Faster TypedArrays: Vector Addition in WebAssembly
by brrrrrm on Sunday January 02, 2022 @01:08PM
(#29771577)
I believe numpy is bounded by Python's interpreter (which runs at about ~1us per dispatch to C function from what I recall while working on PyTorch). The number you get is definitely what I'd expect.
If you use larger arrays (and use the "out" variant: `numpy.add(a, b, out=c)`),
you might get similar total throughput at least.
import numpy as np
N = 1024 * 128
A = np.random.randn(N)
B = np.random.randn(N)
C = np.random.randn(N)
import time
for _ in range(1000):
np.add(A, B, out=C)
iters = 1000
t = time.time()
for _ in range(iters):
np.add(A, B, out=C)
d = time.time() - t
print(f"{iters * N * 3 * 4 / d / 1e9:.2f} GB/s")
Re:Faster TypedArrays: Vector Addition in WebAssembly
by lmeyerov on Sunday January 02, 2022 @01:05PM
(#29771526)
Cool to see this stuff getting out into the wild, long time coming!
What is the current state of these -- do simd wasm ops run in ffox/chrome/edge/safari, and does the data have to be slowly bulk copied back-and-forth from a worker (heartbleed removal of zero-copy ownership transfers)?
We love js typed arrays and columnar analytics -- graphistry contributed the first years of Apache Arrow JS -- but intentionally didn't do wasm for kernels because of this kind of stuff, despite promising internal multicore etc prototypes. A surprise win for us of GPU python/js offloading to the server has been not just scale but perf reliability. Curious how much it has improved in the typical case, as it always made sense on paper!
Re:Faster TypedArrays: Vector Addition in WebAssembly
by brrrrrm on Sunday January 02, 2022 @01:17PM
(#29771700)
Yep, you still need to bulk copy memory around if it already exists.
If you own the memory, though, you can avoid copying (wasm has "import") but you'll need to manage that manually.
I've found the cleanest approach for me is to have modules allocate space for inputs and outputs and then try to get functions to write directly into the input space.
Either way, nothing available is super friendly for user provided arrays or canvas interactions.
Re:Faster TypedArrays: Vector Addition in WebAssembly
by twoodfin on Sunday January 02, 2022 @01:01PM
(#29771466)
I’d be interested to see if the manual loop unrolling is necessary, or if the WASM toolchain would perform that optimization automatically if `len` were a known constant.
Re:Faster TypedArrays: Vector Addition in WebAssembly
by brrrrrm on Sunday January 02, 2022 @01:39PM
(#29771990)
That's a good question, and I think there's some data in the benchmark dump to help answer it.
The tuning logic for the results labeled `wasmblr (tuned X)` sweeps through a couple values for X (the unroll size) and shows the best one. On some browsers (and in node.js) this value goes to 1, which means unrolling isn't found to be necessary for that size.
Mostly, from what I've seen, it tunes to a value greater than 1. So I think generally it's still necessary today.
Re:Faster TypedArrays: Vector Addition in WebAssembly
by olliej on Sunday January 02, 2022 @02:51PM
(#29772976)
Any WASM that runs for more than a few milliseconds is going to be compiled to native code. I do wonder just where the remaining 50% of bandwidth is going. IIRC bounds checking is only be around a 10% penalty in most studies. I guess there are other checks needed in JS (stripping signaling NaNs, etc but I don’t think WASM requires that)
4x unwrap: https://jsfiddle.net/49j7htdz/1/
8x unwrap: https://jsfiddle.net/49j7htdz/2/
That being said, I’m seeing a very similar relative difference in your fiddle to what I see in the benchmark I posted.
No kidding, this can be seriously misleading.
IMO JS (and likely many other JIT VM) benchmarks should run:
- multiple cold starts
- multiple pre-warmed starts (essentially run your bench twice, discard the results, run it again)
- isolated cases
- shared runtime cases with evenly distributed ordering
All of those will reveal different performance characteristics you may want to know about, and people evaluating your results should be able to know about. Cold starts are a worst case scenario of each implementation, warmed should show a more ideal outcome. Isolated will again highlight the perf of each implementation on its own merits, whereas shared runtime should help highlight how interacting with other real-world code affects VM optimization of that implementation.
There are a few clickable benchmark examples in the readme: https://github.com/xpl/what-code-is-faster#what-code-is-fast...
Reminds me of prof. Emery Berger's talk, Performance Matters: https://www.youtube-nocookie.com/embed/r-TLSBdHe1A
Even with a warmup, the unwrapped version is more than 3x faster.
Largely, the use modern javascript and proper benchmarking made the results more consistent with the benchmark posted.
This included
1) avoiding the use of "var" (using let for scoped iterator variables and const for unchanging references)
2) using proper warmup (10% of runs)
3) isolating the benchmarked code into a function that can be invoked within a loop
4) reseting the values of the arrays being added between benchmarks (to prevent any NaNs/infinities from cropping up)
Inserting the loop into a function really speeds it up in Chrome. Also, I have noticed, that for Float32Array, the rolled / unrolled performance is almost identical, but for Uint32Array, unrolled is about 30% faster: https://jsfiddle.net/aq06p94f/
That's pretty opaquely rude, just FYI.
Same machine, JSC (Safari) is 3x faster than Chrome https://jsbenchit.org/?src=c792550e65de1d038b1f24b446c74592
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
I didn’t know about fround but I suspect its primary use case is for trying to catch floating point overflow during double arithmetic, because again precision difference is observable.
I get 1.5mil iterations in numpy on CPU which is on par with typed arrays and much slower than wasmblr.
If you use larger arrays (and use the "out" variant: `numpy.add(a, b, out=c)`), you might get similar total throughput at least.
What is the current state of these -- do simd wasm ops run in ffox/chrome/edge/safari, and does the data have to be slowly bulk copied back-and-forth from a worker (heartbleed removal of zero-copy ownership transfers)?
We love js typed arrays and columnar analytics -- graphistry contributed the first years of Apache Arrow JS -- but intentionally didn't do wasm for kernels because of this kind of stuff, despite promising internal multicore etc prototypes. A surprise win for us of GPU python/js offloading to the server has been not just scale but perf reliability. Curious how much it has improved in the typical case, as it always made sense on paper!
I've found the cleanest approach for me is to have modules allocate space for inputs and outputs and then try to get functions to write directly into the input space.
Either way, nothing available is super friendly for user provided arrays or canvas interactions.
Mostly, from what I've seen, it tunes to a value greater than 1. So I think generally it's still necessary today.