News for hackers. Stuff that matters.

A comment on Bad numbers in the “gzip beats BERT” paper?. Parent comment

Re:Bad numbers in the “gzip beats BERT” paper? by hoosieree on Monday July 17, 2023 @12:08PM (#36759941)
I almost feel like compression and embeddings are duals of each other, but I can't quite articulate it.

Embeddings use fixed-size vectors to minimize the dot product between vectors of similar inputs. Compressors use a variable-length encoding to minimize the overall stored size.

[ Parent | on HN ]
Re:Bad numbers in the “gzip beats BERT” paper? by naijaboiler on Monday July 17, 2023 @07:15PM (#36765409)
they are in a way. both encode representation of larger amounts of information on a smaller footprint.
[ Parent | on HN ]