Lossless LLM compression for efficient GPU inference via dynamic-length float

Re:Lossless LLM compression for efficient GPU inference via ... by vessenes on Friday April 25, 2025 @08:33PM (#43799831)

Thanks Jeff -- can you point me to something written up about rANS? All I find on line is turbulence modeling solutions; I presume this is not what you're referring to.

As we know, quantizations are a critical tool for local LLM runners; RAM is typically the gating factor. Are you aware of other better lossless compression of BF16 weights out there?

The reason I ask is this Dfloat11 seems relatively easy to plug in to existing quantization workflows, but you seem dismissive of the paper -- I presume it's my gap in understanding, and I'd like to understand.

[ Parent | on HN ]

Re:Lossless LLM compression for efficient GPU inference via ... by zorgmonkey on Friday April 25, 2025 @08:43PM (#43799888)

I don't know of any great write-ups unfortunately, but the rANS you're looking for is range asymmetric numeral systems.

[ Parent | on HN ]

Re:Lossless LLM compression for efficient GPU inference via ... by eln1 on Sunday April 27, 2025 @02:29AM (#43809865)

There are lots of materials about ANS, e.g. gathered here: https://encode.su/threads/2078-List-of-Asymmetric-Numeral-Sy...

[ Parent | on HN ]