News for hackers. Stuff that matters.
Re:Overfitted a 900KB Transformer to Compress a 100MB CSV in... by spidy__ on Thursday June 25, 2026 @04:18AM (#48670504)
For the model overhead to become significant enough to eat into the gains, the file size would need to be fairly small, right? I assumed nobody would use this for compressing anything below 100 MB.

I tested with 100 MB files because anything larger takes a long time to evaluate. The actual target was at least 1 GB, and in that case I would use a 100 MB model (Shannon entropy rules).

I also tried it on a 100 MB Photoshop file and was able to compress it down to 45 MB, whereas ZIP could only get it down to 60 MB. So yeah still not losing gains.

There are no comments on this comment yet.