> Which answer do you give? Whatever your software tells you (e.g., 87.14234%) or a number made of a small and fixed number of significant digits (e.g., 87%).

> The latter is the right answer in almost all instances.

and

> Most human beings are happy with a 1% error margin.

While he doesn't say it as a rule outright, the author repeatedly uses a "small and fixed" two significant digits. He later says this:

> You must choose the number of significant digits deliberately.

which I agree with, but is at odds with the "small and fixed" dictum with which he leads off the post.

Fixed doesn't mean 2. If it is statstically and practically significant. It could be 3 or 4.

If you're adjusting the number of significant figures you report based on what is practically and statistically significant then it's no longer fixed.

It doesn't need to be same for all the task, just fixed for a particular task. The creators of benchmark or the community could come up with digits of significance that should be the standard, which is based on expected error range. For imagenet it could be 4, for ARC it could be 2.