Examining Microsoft Edge Browser Performance

Re:Examining Microsoft Edge Browser Performance by pcwalton on Tuesday July 21, 2015 @06:24PM (#9926297)

Heavily related to this, one key insight that I think has been neglected is that JavaScript benchmarks do a bad job of measuring cold code. Most code on the Web is cold, because much Web browsing consists of fresh page loads. Heavily optimizing JITs are exactly what you don't want for cold code: time spent compiling is time you could have been spending running the code. But benchmarks consist of running the same code repeatedly in a long-running loop, which encourages development of sophisticated, highly-optimizing JITs that spend a lot of time in up-front compilation, which end up being totally pointless for JS whose sole purpose is to run once to place an ad somewhere on the page.

It's easy to forget that JavaScript is still primarily a scripting language used to perform high-level application tasks. It's awesome that JS has gone so far low-level, and we should continue to push it as far as it can, but I think the bread and butter that users care about is not so much heavy number crunching but rather quick cold code that needs to run once or twice and get out of the way.

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by haberman on Tuesday July 21, 2015 @09:02PM (#9927061)

> Heavily optimizing JITs are exactly what you don't want for cold code: time spent compiling is time you could have been spending running the code.

I agree with your overall point of cold cold performance being paramount. But I don't think this follows. A heavily optimizing JIT can have very good cold-code performance if the JIT is especially fast and/or it makes especially good decisions about when to JIT.

The important trait is short break-even time: how long after JIT-ting some code that you break even on the investment of JITting time.

When all I knew was the JVM I assumed that JIT-compiling inherently had a long warm-up / break-even time. Now that I've experienced LuaJIT I know this isn't true. I haven't benchmarked the major JIT-ting JS engines but they seem pretty good at this too?

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by pcwalton on Wednesday July 22, 2015 @12:39AM (#9927766)

By "a heavily optimizing JIT" I mean "the specific mode of a larger JIT-enabled engine that compiles slowly but emits good code". So IonMonkey, Crankshaft, SpeculativeJIT/FTL are all JITs that live inside a larger engine that switches between them as needed. In JS land we usually call the individual modes "JITs" even though they may (or may not) share a bunch of code.

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by jgraham on Tuesday July 21, 2015 @06:59PM (#9926481)

I don't know if it's a coincidence that around the time that paper was released Microsoft were considerably behind the curve in terms of js engine performance. The paper also misses the fact that excellent hot-code performance opens up new frontiers for the web like games, photo editing, and similar compute-intensive applications that are not well represented by their sample of popular sites since those are, by definition, limited by what was performant in extant browsers.

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by jcranmer on Tuesday July 21, 2015 @07:14PM (#9926574)

Well, yes and no. Pure JavaScript benchmarks (SunSpider, Octane, Kraken) are all more or less microbenchmarks, at least to the degree that they're testing only one specific kernel of an application (even if Octane and Kraken are justifable since they're pulling largish kernels from real code). A pure JS "cold" benchmark would be an even poorer microbenchmark, since it carries a large presumption that the cold JS is the bottleneck. In real JS code of the stuff you're talking about, you might be able to easily get measurable performance gains by, say, improving FFI between JS and DOM code, or improving the DOM code itself.

A better benchmark would be measuring page load times (or energy usage) or dynamic operations that included such cold JS code, since it would illustrate where performance improvements would be most beneficial.

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by arsa on Tuesday July 21, 2015 @08:32PM (#9926926)

Have you actually looked at some of these benchmarks? For example, Octane contains "Code loading", sunspider has "code decompression" (not true code loading, but at least it does de-minification).

Of course it's important, and all borwsers are developed with this in mind - but it's not possible to create reproducable test suite (with a score given) that will test actual sites - sites change

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by pjmlp on Wednesday July 22, 2015 @06:33AM (#9928735)

> I think the bread and butter that users care about is not so much heavy number crunching but rather quick cold code that needs to run once or twice and get out of the way.

I care about HTML and CSS with as little JavaScript as possible. The best code to optimize is the one isn't there in first place.

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by sanxiyn on Wednesday July 22, 2015 @01:06AM (#9927834)

JSBench is a good benchmark measuring cold performance. I recommend everybody try running it.

It is known that Safari wins this benchmark, by a large margin.

[ Parent | on HN ]

Re:Examining Microsoft Edge Browser Performance by randomfool on Wednesday July 22, 2015 @01:14AM (#9927855)

Unfortunately the way it measures JS perf is questionable- it essentially records all operations during page load then replays them for the benchmark. This has the effect of unrolling all of the loops and disabling many of the optimizations that would have been applied to the original code.

[ Parent | on HN ]