Re:Compression efficiency with shared dictionaries in Chrome
by saagarjha on Wednesday March 06, 2024 @12:19PM
(#39618252)
So Brotli already contains a dictionary that is trained on web traffic. I think the thing here is that Google wants to make sending YouTube 1.1 more efficient if you already have YouTube 1.0, but they can’t put YouTube 1.0 into the browser.
[ Parent
| on HN ]
Re:Compression efficiency with shared dictionaries in Chrome
by hinkley on Wednesday March 06, 2024 @02:54PM
(#39620534)
This is something game devs have been doing for decades.
If you want to delta 1.0 to 1.1 that’s server side work you do once at deployment or build time, not on every request.
Re:Compression efficiency with shared dictionaries in Chrome
by Wingy on Wednesday March 06, 2024 @04:49PM
(#39622070)
What happens when you release 1.2 and someone who has 1.0 visits? Do you generate a delta for every past version at build time?
Re:Compression efficiency with shared dictionaries in Chrome
by patrickmeenan on Wednesday March 06, 2024 @07:13PM
(#39623502)
You determine how far back you want to build deltas for. If you build deltas for the last 3 versions then you can send diffs for those users as well (as long as the dictionary hasn't expired). Or, you could just send the full response just like if dictionaries weren't supported.
Each site can decide what a "good" number of releases to build against based on typical release cycles and user visitation patterns.
Re:Compression efficiency with shared dictionaries in Chrome
by hinkley on Wednesday March 06, 2024 @05:00PM
(#39622211)
You mean if a user who hasn’t visited the site in a year comes back?
They download 1.2 because 1.0 is no longer in their browser cache, that’s what.
The web is easier than games because “files at rest” are much more volatile on the web.
Re:Compression efficiency with shared dictionaries in Chrome
by burnhamup on Wednesday March 06, 2024 @10:46PM
(#39624859)
The patch system I worked with generated signatures of each build. The signature had the hash of each block of the build. The client has the signature for their version (1.0) and they download the signature of the new version (1.2) and diff the two. Then they download each block that has changed.
I think it was the `electron-updater` for my electron app, but I don't quite remember now.
Re:Compression efficiency with shared dictionaries in Chrome
by hinkley on Thursday March 07, 2024 @10:28AM
(#39630265)
It was just a couple years ago that I learned that the Unix compression libraries have a flag to make them “rsync friendly”. They do something with compression blocks to make them more stable across changes. Normally a small change in the middle of a file could change the rest of the output, due to bit packing.
I should really figure out how that works.