News for hackers. Stuff that matters.

A comment on Lossless LLM compression for efficient GPU inference via dynamic-length float. Parent comment

Re:Lossless LLM compression for efficient GPU inference via ... by gojomo on Friday April 25, 2025 @03:28PM (#43797678)
Some might prefer the fidelity of this method's 70% savings over the lossyness of 4-bit quantization's 75%.

And, maybe the methods stack for those willing to trade both costs for the smallest representation.

[ Parent | on HN ]
Re:Lossless LLM compression for efficient GPU inference via ... by svachalek on Friday April 25, 2025 @03:39PM (#43797775)
This is only a 30% savings, which is a cool technical feat but hard to see a use case for.
[ Parent | on HN ]