Re:I benchmarked 8 different ways of representing a 2d grid ...
by shazow on Friday December 31, 2010 @02:13AM
(#2054825)
You're right, I should be running each test at least a few times and taking the average. I found that the results vary a few percent here and there, but the drastic difference between the different datastructures is always consistent and visible.
Re:I benchmarked 8 different ways of representing a 2d grid ...
by mathias on Friday December 31, 2010 @04:00AM
(#2054955)
The other article you’re talking about is this one: http://ejohn.org/blog/accuracy-of-javascript-time/
Note that some interesting information is missing from it (e.g., in IE the smallest unit of measure isn’t always 15 ms — it all depends). If you want to know all about it, check out http://calendar.perfplanet.com/2010/bulletproof-javascript-b....
It appears that execution times change drastically depending on function names and orderings! (This is in both firefox and chrome, on a macbook.) I tried copying the 2d array function and renaming it different things, and found different times, which often caused major differences in its relative order against the original function.
Also, running the test multiple times gives different enough results that orderings change.
Finally, I think the timing code needs some work. I remember John Resign had some posts on this. This is one of them:
http://ejohn.org/blog/javascript-benchmark-quality/
But I can't find another one where he talks about the fact that with very small times, timing functions in javascript give wildly varying values, because they depend on some sort of interrupt (maybe?)...