News for hackers. Stuff that matters.
Re:Differentiable Dithering by londons_explore on Wednesday September 16, 2020 @04:09AM (#24490570)
> A pipedream would be an entirely differentiable image compression pipeline where all the steps can be fine tuned together to optimize a particular image with respect to any differentiable loss function.

Neural Image Compression? https://arxiv.org/abs/1908.08988

Re:Differentiable Dithering by tonic_section on Wednesday September 16, 2020 @07:59AM (#24491736)
Unfortunately you wouldn't have any guarantees on the output of any particular image though, just some reassurances about the expected behaviour over the training set.
Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @11:49AM (#24493793)
> any guarantees on the output of any particular image though

You don't have any guarantees with this non-convex optimization.

I think most of these methods would work OK on out-of-domain data.

Re:Differentiable Dithering by tonic_section on Wednesday September 16, 2020 @12:44PM (#24494408)
In terms of the decoded image, yes - it's very unlikely you would get something substantially different from the original image. But in terms of the bitrate it's not hard to find examples where the compressed bitrate can be several standard deviations above the average bitrate on the training set - see e.g. the last example here: https://github.com/Justin-Tan/high-fidelity-generative-compr...

(Lossy) neural compression methods may also synthesize small portions of an image to avoid compression artefacts associated with standard image codecs, so should definitely not be used in sensitive applications where small details can make a big difference such as security imaging, guarantees or none.

Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @01:25PM (#24494918)
Those are very cool examples and of considerably higher quality+bitrate than when I last tuned into this field half a year ago.

Unrelated, but I actually recognize your name from Github - I guess deep image compression is a pretty small space.

Re:Differentiable Dithering by underanalyzer on Wednesday September 16, 2020 @11:47AM (#24493767)
Yea neural image compression looks pretty neat! The reason I bring up jpeg is because it's so well established and if you come up with a more optimized jpeg (which I'm not really convinced is possible, again a pipedream) you don't have to force people to transition to a new image format. In the end methods like neural style or whatever comes after are probably the better pick but there is a transition period.
Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @11:51AM (#24493819)
> more optimized jpeg

The thing about compression is that there is no single "more optimized" knob - there's a bunch of different tradeoffs.

Want a compression algo that can compress existing images to smaller sizes than JPEG? You can already do that with neural image compression. Want a compression algo that can decode that compressed image in 0.01 seconds? You need JPEG.

Re:Differentiable Dithering by underanalyzer on Wednesday September 16, 2020 @12:19PM (#24494128)
Sorry I could have been more specific. By more optimized jpeg I meant better perceptual quality (again subjective) within the confines of what a jpeg decoder could understand.

To co opt your knobs analogy I imagine each of the steps of a complex image compression pipeline comes with its own knobs each with its own tradeoffs. The dream here would be to tune all those knobs at the same time to optimize some sense of quality in a particular image. Of course huge disclaimer I’m not an image or signal processing expert. It’s also very possible that these “knobs” have been tuned well enough so that even if we optimized them for a specific image the quality difference would not be noticeable.

Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @01:28PM (#24494953)
The problem here is that the JPEG decoder is not differentiable.
Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @11:55AM (#24493859)
> entirely differentiable image compression pipeline

Depending on what is meant by entirely differentiable, this might be impossible without relaxation. ie. you can't differentiate through the quantization step

Re:Differentiable Dithering by tonic_section on Wednesday September 16, 2020 @12:47PM (#24494447)
There are a couple of solutions which work empirically - as you mentioned, one solution is a dithering-like differentiable relaxation where uniform noise is added, which simulates quantization, or just to ignore the quantization operation when taking gradients, essentially treating it as an identity operation in the backward pass.
Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @01:08PM (#24494700)
But how do you optimize the lossless encoding of the quantized latent space? ie. how do you tell the encoder to produce something that can be well encoded, given that the encoding is a bunch of discrete steps.
Re:Differentiable Dithering by tonic_section on Wednesday September 16, 2020 @01:12PM (#24494767)
Usually the lossless encoding is offloaded to a standard entropy coder, e.g. arithmetic, ANS, etc. because these approach the theoretical minimum rate given by the source entropy pretty closely, so there wouldn't be a point building a fancy differentiable replacement.
Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @01:15PM (#24494793)
That makes sense, I don't think I stated my question very clearly: how do you control/optimize the entropy of the latent space?

ie. what stops the network from laundering all of the information for reconstructing the image through a super high entropy latent space that is hard to code but allows it to reconstruct perfectly

e: I guess I should just get up to date by reading some papers

Re:Differentiable Dithering by tonic_section on Wednesday September 16, 2020 @02:15PM (#24495555)
The objective function used in these lossy neural compression schemes usually takes the form of a rate-distortion Lagrangian - the rate term captures the expected length of the message needed to transmit the compressed information and the distortion term measures the reconstruction error. So it wouldn't be able to cheat like in your example, because this would incur a high value of the loss through the rate term.
Re:Differentiable Dithering by uoaei on Wednesday September 16, 2020 @05:12PM (#24497994)
Formulate it in terms of probabilistic programming, and you will essentially be able to do exactly that.
Re:Differentiable Dithering by amelius on Wednesday September 16, 2020 @06:25AM (#24491264)
I fear that you might end up with hallucinations in your images ...
Re:Differentiable Dithering by IanCal on Wednesday September 16, 2020 @07:25AM (#24491538)
Similar to imagining Ryan Gosling is in your background https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a...
Re:Differentiable Dithering by alanbernstein on Wednesday September 16, 2020 @08:03AM (#24491761)
Ryan Gosling? That's quite a stretch.
Re:Differentiable Dithering by 0-_-0 on Wednesday September 16, 2020 @06:51AM (#24491361)
Isn't that a feature?
Re:Differentiable Dithering by MiroF on Wednesday September 16, 2020 @11:54AM (#24493846)
It's not trained to generate faces, so I don't think so.