News for hackers. Stuff that matters.
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by atorodius on Friday June 26, 2020 @12:02PM (#23653490)
Hi, author here! This is the demo of a research project demonstrating the potential of using GANs for compression. For this to be used in practice, more research is required to make the model smaller/faster, or to build a deticated chip! Currently, this runs at approximately 0.7 megapixel/sec on a GPU with unoptimized TensorFlow. Happy to answer any questions.
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by cs702 on Friday June 26, 2020 @03:33PM (#23656022)
Great work. Thank you for making it so accessible online and sharing it on HN!

The first thought I had, after looking at some of the examples, is that it should be fairly trivial to train or finetune the compression network to hallucinate missing detail with "BPG-style" or "JPG-style" smoothing artifacts by feeding compressed images as "real" samples to the adversarial network during training/finetuning.

I wonder if doing that would make it possible to achieve even greater compression with loss of detail that is indistinguishable from traditional compression methods and therefore still acceptable to the human eye.

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by gtoderici on Friday June 26, 2020 @04:44PM (#23656813)
What you suggest has already been done: train a neural network with the output of BPG or JPEG, and ask to reconstruct the input with just the decompressed pixels being available.

It definitely is a valid approach but the limitation is that if the network needs some texture-specific information that cannot be extracted from the decoded pixels, it can't really do much.

There were approaches where such information was also sent on the side, which yielded better results, of course.

The field is wide-open and each approach has its own challenges (e.g., you may need to train one network for quantization level for example if you're going to do restoration).

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by ksec on Friday June 26, 2020 @12:20PM (#23653715)
>Currently, this runs at approximately 0.7 megapixel/sec on a GPU with unoptimized TensorFlow.

Forgive me, is that in encoding or decoding? And size of library required to decode it? ( Edit: Looks like answered below )

It was only an hour ago I was looking at some pictures encoded with VVC / H.266 reference encoder, it was better than BPG / HEVC, but this HiFiC still beats it by quite a bit.

This whole thing blows up my mind on limitation I thought we had on image compression.

May be I should take back my words on JPEG XL is the future. This seems revolutionary.

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by gtoderici on Friday June 26, 2020 @12:25PM (#23653774)
(coauthor here) The 0.7 megapixels/sec is PNG decode (to get input)+encoding+decoding+PNG encoding (to get output we can visualize in a browser) speed.

Thanks for your kind comment!

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by JyrkiAlakuijala on Wednesday July 08, 2020 @01:43PM (#23772261)
JPEG XL and HiFiC are developed in the same team. JPEG XL is just positioned for slightly less compute than HiFiC.
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by keyi on Friday June 26, 2020 @04:53PM (#23656903)
Have you compared the result with latest image compression formats, such as AVIF[0]?

[0]: https://aomediacodec.github.io/av1-avif/

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by gtoderici on Friday June 26, 2020 @05:01PM (#23657004)
We haven't specifically compared to AVIF, which as far as we know is still under development. We'd be happy to compare, but it's unlikely that we'd learn much out of it. As far as we know, AVIF is better by <100% than HEVC, but we're comparing against HEVC at 300% of the bitrate.

Of course, we'd be happy to add any additional images from other codecs if they're available.

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by bscphil on Saturday June 27, 2020 @02:12AM (#23659865)
I would add JPEG-XL in addition if you're looking for suggestions for other codecs to compare to. It's very competitive with AV1 and beats it, in my opinion, at higher bitrates.

Admittedly, you're not likely to learn much from this that is useful for your research, but most of the interest from people clicking on this is probably wanting to see the latest developments in image compression.

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by petermcneeley on Friday June 26, 2020 @12:22PM (#23653745)
These are truly amazing results. Looking closely at the results vs original it would appear that much of the details are very different at almost a noise level. Is the perceptual evaluation allowing for these similar but completely different noise details?
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by gtoderici on Friday June 26, 2020 @12:27PM (#23653792)
(coauthor here) We used an adversarial loss in addition to a perceptual loss and MSE. None of these work super-well when the others are not used.

The adversarial loss "learns" what is a compressed image and tries to make the decoder go away from such outputs.

The perceptual (LPIPS) is not so sensitive to pure noise and allows for it, but is sensitive to texture details.

MSE tries to get the rough shape right.

We also asked users in a study to tell us which images they preferred when having access to the original. Most prefer the added details even if they're not exactly right.

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by nbardy on Friday June 26, 2020 @02:40PM (#23655406)
> The adversarial loss "learns" what is a compressed image and tries to make the decoder go away from such outputs.

Could you expand on this point

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by atorodius on Friday June 26, 2020 @03:25PM (#23655937)
The idea is that any distortion loss imposes specific artifacts. For example, MSE tends to blur outputs, CNN-feature-based losses (VGG, LPIPS) tend to produce gridding patterns or also blur. Now, when the discriminator network sees these artifacts, those artifacts very obviously distinguish the reconstructions from the input, and thus the gradients from the discriminator guide the optimization away from these artifacts. Let me know if this helps!
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by fiddlerwoaroof on Friday June 26, 2020 @12:32PM (#23653857)
Yeah, it seems like the HiFiC version has the same large-scale structure, but the details are sort of "regenerated": e.g. it's as if the image format says "there's some hair here", and the decoder just paints in hair.
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by gtoderici on Friday June 26, 2020 @12:39PM (#23653952)
That was exactly the goal of the project! Basically if the size doesn't allow to have detail, we need to "hallucinate it". This of course is not necessary if there's enough bandwidth available for transmission or enough storage.

On the other hand, in our paper we show that some generated detail can help even at higher bitrates.

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by belval on Sunday June 28, 2020 @07:50AM (#23668171)
Hi, could you give a general comparison vs CLIC workshop[1] submissions at CVPR? It seems to be the same end goal and CLIC was even sponsored by Google so I am a little puzzled as to why I didn't see your work there.

EDIT: You mention CLIC2020 in your paper so I'm a bit more confused.

Another question would be,what are your thoughts on the future of learned image compression and ideas for future work?

[1] https://www.compression.cc/

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by magicalhippo on Friday June 26, 2020 @05:11PM (#23657086)
Looks really interesting, thanks for sharing!

One silly question, the red door example image... why is the red so saturated in all the compressed versions vs the original?

edit: Ah seems to be some kind of display issue in Firefox. When saving the images and comparing the saturation level is roughly equal.

Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by atorodius on Saturday June 27, 2020 @04:21AM (#23660328)
Interesting, I did not notice this before. This is likely due to the original having some color profile attached. Not sure why this only renders differently in Firefox.
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by UnbugMe on Friday June 26, 2020 @02:28PM (#23655272)
The demo images used on hific.github.io appear to be part of the datasets used to train the system. In another comment you say the trained model is 726MB. The combined size of the training datasets appear to be about 8GB zipped. Is the currently trained model usable on images that are not part of the training datasets? with output of similar quality and size?
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by atorodius on Friday June 26, 2020 @02:36PM (#23655355)
Hi- no the images on the demo page are not part of the training set, they are only used to evaluate the method. Arbitrary images of natural scenes will look similar at these bitrates. We‘ll release trained models and a colab soon!
Re:Show HN: HiFiC – High-Fidelity Generative Image Compressi... by UnbugMe on Friday June 26, 2020 @02:50PM (#23655518)
Thank you and my apologies. I should have read the pdf more carefully where the distinction between the training data and evaluation datasets is described

Our training set consists of a large set of high-resolution images collected from theInternet, which we downscale to a random size between 500 and 1000 pixels, and then randomlycrop to256×256. We evaluate on three diverse benchmark datasets collected independently ofour training set to demonstrate that our method generalizes beyond the training distribution: thewidely usedKodak[22] dataset (24 images), as well as theCLIC2020[43] testset (428 images), andDIV2K[2] validation set (100 images).