Re:Shannon’s Source Coding Theorem (Foundations of Informati...
by exhaze on Sunday June 18, 2023 @11:54AM
(#36381363)
A practical way I used to grok this is to try to zip up an mp4 or a jpeg. You’ll notice the size doesn’t really change. That’s because these formats already applied entropy coding compression techniques and have compressed the information very close to the Shannon entropy limit.
[ Parent
| on HN ]
Re:Shannon’s Source Coding Theorem (Foundations of Informati...
by CorrectHorseBat on Sunday June 18, 2023 @01:36PM
(#36382378)
JPEG XL supports reversible JPEGs transcoding in ~20% less space, which wouldn't be possible if JPEG was very close to the Shannon entropy limit.
Re:Shannon’s Source Coding Theorem (Foundations of Informati...
by Panzer04 on Sunday June 18, 2023 @08:13PM
(#36385861)
JPEG is lossy though - I'm pretty sure these theorems only really apply to lossless compression, because if you allow lossy compression you can do all kinds of things to different t types of data (pictures being the classic case of permitting pretty lossy compression without much noticeable quality degradation)
Re:Shannon’s Source Coding Theorem (Foundations of Informati...
by CorrectHorseBat on Sunday June 18, 2023 @11:14PM
(#36386936)
Maybe I wasn't clear enough: JPEG -> JPEG XL -> JPEG is not lossy, you get exactly the same bits back, but the JPEG XL file is 20% smaller.
Re:Shannon’s Source Coding Theorem (Foundations of Informati...
by Panzer04 on Monday June 19, 2023 @10:27PM
(#36399009)
Oh, very interesting. I'll have to look into it.
Re:Shannon’s Source Coding Theorem (Foundations of Informati...
by pizza on Sunday June 18, 2023 @09:00PM
(#36386099)
There's a distinction to be made here, order-0 entropy versus higher order entropy. Order here just means how long a chain of symbols that is considered, where order-0 means just count each symbol at a time, versus eg order-1 where you consider chains of two symbols. Like the other commenter says, JPEGXL can recompress JPEGs losslessly to be around 20% smaller by exploiting higher order patterns (eg via move-to-front, lz77-ish matches, and the MANIAC tree context modeling, versus JPEG's RLE (which is "sort of beyond order-0") combined with Huffman coding (which is order-0))