Re:High Performance Erlang – Finding Bottlenecks in a CouchD...
by siscia on Monday February 15, 2016 @01:15PM
(#11104917)
Just being extremely pedantic here, but for the least statistician of us I believe is important.
Usually just have lower numbers in your benchmark doesn't actually means that your software runs faster, you should run a statistical test and see.
In this case, if we run a standard test the confidence for improved performance is basically 1, and it is definitely accepted.
However the claim of performance improved of 8%, well the confidence for that claim is of around ~83%, which is quite big, but lower than 95% usually expected in peer-reviewed journals.
Usually just have lower numbers in your benchmark doesn't actually means that your software runs faster, you should run a statistical test and see.
In this case, if we run a standard test the confidence for improved performance is basically 1, and it is definitely accepted.
However the claim of performance improved of 8%, well the confidence for that claim is of around ~83%, which is quite big, but lower than 95% usually expected in peer-reviewed journals.
Said so, I really enjoy the post :)