Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB

Re:Overfitted a 900KB Transformer to Compress a 100MB CSV in... by andai on Thursday June 25, 2026 @11:51PM (#48682143)

I was working on a multiplayer game a while ago, and one of the iterations of the netcode was "thin client" where clients just sent input, server simulated the game, and it dumped world state onto the pipe at 60hz. I didn't ship that version but I estimated a $3000 bandwidth bill with that approach!

I started looking into diffing the state, compression, etc... until I realized, wait a minute! My player movement is linear so I only need a packet for start and stop! And so I achieved near infinite efficiency improvement :)

I think the word is... a specialized solution can beat a general one.

Also, "remembering what the program actually needs to do, and just making it do that"... I de-pessimized the netcode: https://youtube.com/watch?v=pgoetgxecw8

[ Parent | on HN ]

Re:Overfitted a 900KB Transformer to Compress a 100MB CSV in... by purple-leafy on Friday June 26, 2026 @12:45AM (#48682417)

$3000 bill wow!!

Clever insight :) yes a specialised solution usually wins! Good effort

Did you end up publishing your game?

[ Parent | on HN ]

Re:Overfitted a 900KB Transformer to Compress a 100MB CSV in... by andai on Friday June 26, 2026 @05:50PM (#48692432)

My game has been in development hell for several years. I've worked out most of the single-node issues and now I'm working out how to scale it to multiple nodes. (It is embarrassingly parallel, but I am embarrassingly stupid!)

Meanwhile I'm learning how to ship a trivial single player game -- I went with Pong, and it turns out polishing Pong and getting it ready for mass consumption is a surprisingly labor-intensive endeavour (yes, even with the fabled miracle machine assistants!)

But, it should probably be online by the end of July. Like, for real this time :)

What are you working on?

[ Parent | on HN ]

Re:Overfitted a 900KB Transformer to Compress a 100MB CSV in... by purple-leafy on Saturday June 27, 2026 @04:41AM (#48696356)

I hope you can get around to shipping!

I’m working on a game engine/platform and my first game on the engine.

Lots of compression work so that I can fit the games into qr codes which I’m super stubborn about achieving and finally achieved for 30 minute+ games for N number of players

[ Parent | on HN ]