News for hackers. Stuff that matters.
Re:Real world analysis of google's webp format versus jpg by lysium on Friday October 01, 2010 @08:30AM (#1746807)
Unfortunately, webp claims to have same image quality with smaller file size, but the 'real world analysis' compares images quality at the same file size.

Further, image quality is measured in terms of mean RGB or value difference, which is very technical and does not matter to the human eye (a lot). For example, in the portrait, the last picture, the JPEG artifacts in the person's face really hurt more than the blurring of webp, yet both have about the same statistical mean values.

Last, without the original lossless image (at hand), it is hard to tell which lossy encoder is better. Again with the portrait picture as an example, you have to download the provided lossless image to see that webp blurred the face too much.

Still, better than other quick-and-dirty 'analysis' I've seen so far.

Re:Real world analysis of google's webp format versus jpg by jfager on Friday October 01, 2010 @09:08AM (#1746876)
1. How else would you construct an objective test? Creating images of the same quality and then comparing their file sizes seems prohibitively difficult if not impossible. This test, otoh, implies the same information (minus the exact difference in file size), but is pretty easy to set up.

2. Those were the reported numbers, but it's not like that was the only data point discussed in the article. Every single photo had deeper analysis and included subjective evaluation. He even directly made the point that you did about the last photo.

3. Loading megabytes of photo data into folks browsers by default doesn't seem worth the benefit (the page is already sluggish). If you're interested, the originals are available for download.

Re:Real world analysis of google's webp format versus jpg by sp332 on Friday October 01, 2010 @09:25AM (#1746917)
There's more than one "objective" test for image quality, and some of them are closer to human perception than PSNR. The trouble with comparisons like this is that the JPG encoder might be tuned to minimize PSNR, but if the WebP encoder is tuned to optimize for something else, of course it won't perform as well when you compare the PSNR!

More info on the issues with this type of benchmark from Jason Garrett-Glaser (a.k.a. Dark Shikari), an x264 developer: http://x264dev.multimedia.cx/?p=458

Re:Real world analysis of google's webp format versus jpg by jfager on Friday October 01, 2010 @09:36AM (#1746946)
That's fine, I agree. My point was not "His metric is the most awesome, shut up", it was "Despite only giving one kind of number, the author still included other analysis and subjective opinion in the writeup for each photo, so it's not as bad as you're trying to imply".
Re:Real world analysis of google's webp format versus jpg by sp332 on Friday October 01, 2010 @09:47AM (#1746973)
You asked How else would you construct an objective test? And my answer is, just make sure you're being fair in measuring the same thing the encoders are optimising for.
Re:Real world analysis of google's webp format versus jpg by jfager on Friday October 01, 2010 @09:51AM (#1746984)
That was in response to the complaint that the post looked at images of the same size and compared their quality, rather than looking at images of the same quality and comparing their size. If you know of a practical way to objectively do the latter, please share.
Re:Real world analysis of google's webp format versus jpg by scott_s on Friday October 01, 2010 @11:49AM (#1747398)
If I only test the optimal circumstances for a new thing, then I have done a terrible job at trying to understand the consequences of adopting it.
Re:Real world analysis of google's webp format versus jpg by sp332 on Friday October 01, 2010 @05:18PM (#1748798)
On two occasions I have been asked,—"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" .... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

--Babbage (1864), Passages from the Life of a Philosopher, ch. 5 "Difference Engine No. 1"

You can't reasonably use an encoder optimized for PSNR and then "ding" it for producing output with bad SSIM, or vice-versa.

Re:Real world analysis of google's webp format versus jpg by scott_s on Friday October 01, 2010 @05:41PM (#1748889)
I'm not familiar enough with the image encoding algorithms (and the tradeoffs they are faced with) to comment on that, specifically. But I've done enough research with systems in general to disagree with your statement. No matter what I optimize a system for, it's always fair for someone to test it under adverse conditions. If we're going to adopt something, we need to know its limitations.

Further, my understanding of Google's intent with webp is that it is being offered as a replacement to jpeg. In that case, even if it's optimized for one thing, it's not just fair but necessary to see how it works under all other things.

Re:Real world analysis of google's webp format versus jpg by lysium on Friday October 01, 2010 @12:15PM (#1747491)
Well, if you test an algorithm that claims to have same image quality with smaller file size, you should compare images of the same quality and look at the files size, not the other way round.

Besides, I don't see why you need megabytes of photo data, regular internet photo data would have sufficed (for which the authors say they have developed webp anyways).

I agree with your point that he included subjective evaluation, though.

Re:Real world analysis of google's webp format versus jpg by eru on Friday October 01, 2010 @09:10AM (#1746890)
About 1: This test looks at the dual claim, and that's more or less justified. (Look up duality in linear optimization for the definition. It's useful.)
Re:Real world analysis of google's webp format versus jpg by jjcm on Friday October 01, 2010 @03:42PM (#1748326)
I'm the author of the article. You make an excellent point here. It's very hard to quantify image quality. While true, you do need to have the lossless source files (http://jjcm.org:8081/webp) if you want to do a true comparison from the original, but in the real world I'd venture to guess you wont have those; that's why I felt it was unnecessary for most readers.