> I believe that's not even using the streaming compilation mentioned in the article

That's correct. Streaming compilation would finish earlier, but might actually benchmark more slowly because you'd be adding in the time that the compiler is idle and waiting for the network to catch up.

Preloading the .wasm file in the test lets us measure just the speed of the compiler, independent of the network.

An interesting benchmark would be the total energy usage of the CPU while loading a page in each browser. The compiler managing to idle waiting for network packets should theoretically allow some of the CPU cores to enter sleep states. (Not all, since others are still busy rendering the page, and at least one is busy doing whatever non-DMA kernel bits are involved with receiving the network packets.)

It should be the case that if the same amount of work is done, then the energy used will be the same. If it takes less work to compile the web assembly, then less energy (holding all other parameters the same). If you have to idle a CPU, then you probably use more energy (holding all other parameters the same) i.e. because you will spend more time and accomplish the same amount of real work (but waste energy on the idled core, albeit waste very little, accomplishing extra, but non-productive, work). Cannot let some other CPU parameter changes as a result of cores being idled (e.g. frequency gets boosted on non-idle cores as a result of dynamic frequency scaling with idle cores) to run this experiment. Thinking about CPU energy use is interesting :)

I don't think that's true; isn't the relationship between clock speed and power non-linear?

The real killer with energy usage in browsers is idle wake ups per second. If you've got a lot of tabs open and they're all running timers, waking up, hitting the network, etc. then they're keeping the CPU from going into low power state and thus wasting a lot of energy.

Even though I'd prefer to use Firefox I tend to stick with Safari due to the battery life advantage which really shows when you open a lot of tabs.

Indeed the most dramatic improvement I saw in battery life was when macOS and Safari started cooperating around timer coalescing using App Nap.

That's for max frequency on a given process node. Power scales with voltage squared. But that doesn't say anything about wasted power. And dynamic scaling screws that up in modern chips.

I believe I could summarize things by saying the only way you can really save energy* doing the same work+ is by using a different semiconductor process (either power/leakage-reduction-focused or smaller).

* For serious values of "energy"

+ Where the same work is not always true for a given task, if one optimizes an algorithm

Can you explain the voltage squared thing? To me, power = voltage * current.

Even for purely resistive loads, power is proportional to the square of the voltage:

  P = V * I

But,

  I = V / R

So,

  P = V * (V / R)
    = V^2 / R

For chips, power scales with voltage squared. Is also true that P=IV (since both are true, these observations cannot be in contradiction). Apparently, for chips, the current must be proportional to voltage also. Glossing over some details, turning on (off) a transistor is the same as charging (discharging) a capacitor. The energy stored on a capacitor is 1/2 C V^2. If you turn on and off the transistor periodically (say with frequency f) you use 1/2 C V^2 energy f times per second (energy per unit time is power). Normally the capacitance is ignored when discussing how power changes because for a given design the capacitance is a fixed quantity.

Running a processor at higher frequency also requires increasing voltage, which increases effective capacitance by that same formula.

That's not the primary cause of the power = frequency^2 rule, but actually adds a factor on top of that.

I think it is linear for frequency and non-linear for voltage, i.e. P~fCV^2. But in many current CPUS, the feature that adjusts frequency also adjusts voltage. That's why I stipulated that, for my comments to be true, such shenanigans as dynamic frequency (and voltage) scaling must be "turned off." I think the OP was asking, what happens to CPU energy if you load the web page with and without the optimized compilation. The OP was interested in core sleep states, but I think that dynamic frequency scaling is a confounding factor. It would be interesting to see the measurements w/ and w/out that feature perhaps.

To increase the frequency, you also have to increase the voltage so that the transistors charge faster, otherwise they won't be able to switch in the shorter time.

I ran the tests on my Nexus 5X running stock Android 8.1.

Chrome: WebAssembly.instantiate took 12935.5 ms (1 MB/s)

Firefox Nightly: WebAssembly.instantiate took 1223.1 ms (10.1 MB/s)

Yikes, one order of magnitude in difference.

Similar for me, a touch over 10x

FF: WebAssembly.instantiate took 280.3 ms (44.2 MB/s)

Chrome: WebAssembly.instantiate took 3022.4 ms (4.1 MB/s)

mine was even a bit more of a change.

Chrome: WebAssembly.instantiate took 13692.8 ms (0.9 MB/s)

Firefox: WebAssembly.instantiate took 330.8 ms (37.4 MB/s)

Safari 11.0.2: 4.7mb/s Chrome: 1.2mb/s

This is on MacOS 10.13.2. I'd love to run Firefox but the battery savings and reduced heat from using Safari makes it too hard to pass up in this regard.

Chrome Dev: WebAssembly.instantiate took 2254 ms (5.5 MB/s)

Firefox Nightly: WebAssembly.instantiate took 158.2 ms (78.3 MB/s)

Edge 41.16299.15.0: WebAssembly.instantiate took 99.2 ms (124.8 MB/s)

I did not expect Edge to be even faster.

As stated elsewhere in this thread, Edge compiles WebAssembly lazily, so it's basically skipping the test: https://news.ycombinator.com/item?id=16170496

Is the runtime performance similar to the startup performance measurement?

I'm running Firefox 57, and even here, Firefox is significantly faster; 3182.3ms (3.9mb/s) in Firefox, 7575ms (1.6mb/s) in Chrome 63.

Chrome: WebAssembly.instantiate took 22861.1ms (0.5mb/s)

Nightly:WebAssembly.instantiate took 1825.3ms (6.8mb/s)

Mine was a laptop in low power mode. Yours was much faster all around.

My old i5 Thinkpad L450:

Firefox 57: 5053.8 ms (2.4 MB/s)

Firefox Nightly 59: 454.6 ms (27.2 MB/s)

Chrome 63: 9034.9 ms (1.4 MB/s)

Wow, it's over x10 faster...

My times are similar. I also ran on Safari which is faster than Chrome but slower than Firefox.

Where did you even find that repo? Was that mentioned in the article or is this your code?

Linked in the article under "give it a try" in the sixth paragraph.

The repo itself is at https://github.com/lukewagner/test-tanks-compile-time

In Firefox 58 beta I get: WebAssembly.instantiate took 200.9 ms (61.6 MB/s)

In Safari 11.0.2 I get: WebAssembly.instantiate took 2885.9 ms (4.3 MB/s)

In Vivaldi 1.13.1008.40 I get: WebAssembly.instantiate took 7719 ms (1.6 MB/s)

I tried Chrome Canary, Firefox Nightly and Edge. Edge was the fastest here! But I believe they are using streaming compilation already.