News for hackers. Stuff that matters.
Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by kalleboo on Thursday May 05, 2011 @12:47PM (#2518007)
I was hoping they'd discuss how SunSpider doesn't test the DOM, Canvas or other things that web apps commonly do. I haven't felt my apps have really been performance constrained by basic JS language performance, but rather DOM tree searches, element creation, innerHTML parsing, etc. Benchmarking a JavaScript Raytracer and encryption engine is a noble cause for advancing JS going forward (especially with projects such as node.js gaining momentum), but it doesn't feel very relevant to me, right now, today.

What other benchmarks are there? Apple's Sunspider, Google's V8 Benchmark Suite, Mozilla's Kraken, that JSLint-based one... Does anyone know of a good link where someone has deconstructed all these JS benchmarks and what parts of a browser they actually test?

Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by othermaciej on Thursday May 05, 2011 @02:38PM (#2518451)
SunSpider was deliberately designed to test JS only. What's regrettable is that folks started interpreting JS performance as the primary or even sole metric of browser performance. JS performance has improved by more than an order of magnitude in most browsers since JS was first created, so it's even less likely to be a bottleneck.

It's tricky to make good benchmarks. One interest area of new browser-based benchmarks is ones that focus on graphics. Here are some recent benchmarks that focus on graphics, which I think is an up and coming hot area for web applications, particularly games:

JSGameBench - http://www.facebook.com/note.php?note_id=491691753919&id...

MainInBlue AnimationBenchmark - http://www.themaninblue.com/writing/perspective/2010/03/22/

GUIMark2 - http://www.craftymind.com/guimark2/

Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by rimantas on Thursday May 05, 2011 @01:38PM (#2518206)
Testing pure JS without DOM operations is the reason why I don't pay that much attention to all the bragging about speed: it's mostly irrelevant to what I do. There is however http://dromaeo.com/ which does test for DOM operations.
Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by kenjackson on Thursday May 05, 2011 @12:51PM (#2518029)
Yes, running a benchmark that doesn't test the right things 50x more doesn't really fix the core issue.

I am curious though why IE gets slower. At worst it should run about the same speed.

Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by mbelshe on Friday May 06, 2011 @12:46PM (#2521520)
The great thing and the worst thing about benchmarks is that vendors optimize to them :-)

IE9 claims speed, but the only benchmark it performs better than other browsers on is the SunSpider benchmark. If it is so fast, why does it only do well on a single benchmark? It has already been documented that trivial changes to SS make IE9's performance change dramatically: http://blog.mozilla.com/rob-sayre/2010/11/16/reporting-a-bug...

Personally, I believe SunSpider is just too-easy-to-game. Historically, this problem comes up now and then with compiler vendors and benchmarks. The only solution is to have long-running benchmarks which exercise many parts of the JS engine so that gaming is not so easy.

Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by kenjackson on Friday May 06, 2011 @01:33PM (#2521702)
Actually the IE9 SS issue was a bug. It's since been fixed and no longer displays this weird behavior in IE9.

IE9 also does well on other benchmarks too like the Facebook graphics benchmark, where I think the only browser to beat it was FF4.

Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by mbelshe on Friday May 06, 2011 @01:44PM (#2521751)
There were multiple issues. There was a straight up bug, which was fixed. But the perf delta remains.
Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by mbelshe on Friday May 06, 2011 @02:44PM (#2521988)
You can see this for yourself - run the test referenced in Rob's post above. IE9, right now, reports a 13-14x slowdown on that test with trivial changes.
Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by kenjackson on Friday May 06, 2011 @02:33PM (#2521946)
I don't believe so. I had checked this and the perf issue went away. Maybe you're referring to a different issue. Can you post a link with perf data?
Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider by andrewcooke on Thursday May 05, 2011 @04:32PM (#2518797)
postponed gc?