News for hackers. Stuff that matters.
Re:Macro-Benchmark with Django, Flask and AsyncIO by korzun on Wednesday February 25, 2015 @02:56PM (#9109125)
Flawed benchmarks. If target produces errors during the test, the result should be invalidated immediately.

You can't compare data from failed result to another failed result or a normal one.

In instances where flask/django started to output errors, the test case should have been adjusted until they could have completed operations. Otherwise you can't compare the results, since you have no baseline.

What this test tells me now is XY gives errors for this test while Z seems to process it. That's not a benchmark of any sort.

There are no comments on this comment yet.