Re:Google Criticizes JavaScript Benchmarks, Modifies Sunspider
by ErikCorry on Thursday May 05, 2011 @02:49PM
(#2518492)
I agree, but I do have one nitpick: I've seen this suggestion several places, but I don't really think it's accurate to say that JS engine improvements are levelling off. See http://blog.chromium.org/2010/12/new-crankshaft-for-v8.html Improvements to SunSpider scores, on the other hand, are levelling off, as you note.
as js engine improvements are starting to level off, the tradeoffs that have to be made become more complex. optimizing for one case can cause a regression in another (just take a look at the heuristics used for when tracing should kick in for trace-/jaegermonkey). at some point, insisting on monotonically decreasing times for a test that takes a quarter of a second becomes pretty much akin to the megahertz or megapixel races.