It doesn't need to be same for all the task, just fixed for a particular task. The creators of benchmark or the community could come up with digits of significance that should be the standard, which is based on expected error range. For imagenet it could be 4, for ARC it could be 2.