> Showing a smaller file that "looks the same" is easy, but deceptive.
It's much better to show same-sized files and let the viewer assess their quality (this is what the comparison I linked does), but there are deceptive ways to do even this.
See also this great old blog post about doing comparisons for video codecs: https://web.archive.org/web/20141103202912/http://x264dev.mu...
Samsung turns their screens and phone cameras to “vivid” processing by default (they do provide a “natural” toggle, to their credit).
I think over 90% of people don’t realize what they are seeing is not close to reality, and instead ask people with other phones, especially iPhones, why their screen or photo looks so washed out.
There’s many faults to BFDLs, but I love Apple providing “vivid” processing as an option and “natural” as the default. Sometimes the masses just don’t know what’s actually good for them.
For instance, at medium quality Jxl seems to be better at preserving fine details and structure like the mark on the door, the traces on the lower part of the bridge, but avif appears to be better at preserving clarity of complex details, like far-away windows, cars, a tennis racket.
Sharp edges are just one texture in an infinite range of textures, and AVIF looks like it constructs everything out of sharp edges in a way that's really obvious to the eye at all compression levels.
With JPEG XL I can at least tell what's missing, or too artifacted to make out. With AVIF you have no idea what has been completely erased.
In case you actually wanted to compress images specifically for viewing when zoomed in, you should use different codecs, or higher quality, or configure codecs differently (e.g. in classic JPEG make quantization table preserve high frequences more).
But for a benchmark that claims to compare codecs in general, you should only use normal viewing distance. Currently it's controversial whether the norm is still ~100dpi or whether it should be the "Retina"/2x resolution, but definitely it's not some zoomed-in 5dpi.
Non-blind n=1 human test can be just as flawed. What you like in a particular scenario is not representative of codecs' overall performance.
Testing with humans requires proper setup and a large sample size (which BTW JPEG XL has done!)
The problem is that these codecs are close enough in performance to be below "noise" level of human judgement.
You will not be able to reliably distinguish q=80 vs q=81 of the same codec, even though there is objectively a difference between them. And you can't lower quality to potato level to make your job easier — that changes the nature of the benchmark.
People also just differ in opinion whether blurry without detail is better than detailed but with blocking or ringing. If you ask people to rate images, stddev of their scores will be pretty large, so wide that scores of objective metrics can fit in 1-sigma.
People also tend to pick "nicer" image, rather than the one that is closer to the original. That's a bit different task than codecs aim for.
Codecs can allocate more or less of the file to color channels, so you can get different conclusions based on e.g. amount of bright red in the image.
So testing is hard. Plenty of pitfalls. Showing a smaller file that "looks the same" is easy, but deceptive.