If the test set of images really were 50/50 and the human judges weren't told that, then they were effectively given inaccurate priors, which would obviously reduce their accuracy.