News for hackers. Stuff that matters.

A comment on Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB. Parent comment

Re:Overfitted a 900KB Transformer to Compress a 100MB CSV in... by gravypod on Thursday June 25, 2026 @11:56PM (#48682164)
Since you know the size of the file beforehand you may be able to overfit some kind of text diffusion model instead of a transformer? May allow you to partially correct the model output using some other method and then fill in the blanks that were wrong from previous generations.
[ Parent | on HN ]
Re:Overfitted a 900KB Transformer to Compress a 100MB CSV in... by spidy__ on Friday June 26, 2026 @02:12AM (#48682926)
Oh, sounds interesting. I hadn't considered using a diffusion model for this. My current approach generates the document byte by byte with an autoregressive transformer, so I'm curious how a diffusion model would improve memorization or reconstruction quality.

Can you point me to something that i can read? I really wanna try this approach , diffusion model does sounds interesting for compression.

[ Parent | on HN ]