Re:Making all your integers positive with zigzag encoding
by nine_k on Friday November 25, 2022 @07:02PM
(#33748351)
The essence of the trick: make the sign bit the least significant bit. Then all the leading bytes for integers of small magnitude remain zeros (not 0xFF like in two-complement integers), and compression becomes easy.
[ Parent
| on HN ]
Re:Making all your integers positive with zigzag encoding
by krackers on Friday November 25, 2022 @08:20PM
(#33748872)
This is an amazing explanation, and immediately makes the formula `(val<<1) ^ (val>>31)` intuitive, compared to trying to interpret it as mapping x -> 2|x| - 1.
Re:Making all your integers positive with zigzag encoding
by 10000truths on Friday November 25, 2022 @10:07PM
(#33749498)
In other words, a left rotation by 1 bit.
Re:Making all your integers positive with zigzag encoding
by pifm_guy on Saturday November 26, 2022 @02:29AM
(#33750672)
No. From the typical 2's compliment number encoding used on all modern computers, it is not a simple rotation.
Think about the encoding of minus 1 for an example that differs...
Re:Making all your integers positive with zigzag encoding
by 10000truths on Sunday November 27, 2022 @02:50PM
(#33766087)
Argh, you're right. Classic mistake of tunnel-visioning on a subset of the domain.
Re:Making all your integers positive with zigzag encoding
by hackcasual on Saturday November 26, 2022 @04:46AM
(#33751350)
I used this trick for delta compression for an ArrayBuffer backed data exchange format. Since it was getting sent from a web server, we got gzip for free, but could still shrink data sizes by filtering it in such a way as to make it easier to compress. Delta compressing correlated data points yields a lot of small, but randomly distributed positive/negative values, so by zig-zag encoding them but saving as 32bit values, you end up with a lot of runs of 0's, which get compressed very efficiently, and only require a quick pass through the data to restore the original values
Re:Making all your integers positive with zigzag encoding
by drudru on Monday November 28, 2022 @12:21AM
(#33770086)
Came here to say the same thing. When I first saw the protobuf zigzag encoding, I saw the same pattern. At first, I thought I must be wrong because they aren't describing it this way. Nope, it really is that simple.
In earlier computer systems (pre 1980s), this kind of variety was more common. In my career, I have only been exposed to twos-complement or float formats. I would bet that this is true for many other engineers as well.
I'm glad the protobuf folks used it. I'm also glad Dr. Lemire wrote an article about it. It is another clever trick to have in the toolbox.