News for hackers. Stuff that matters.
Re:Lossless LLM compression for efficient GPU inference via ... by jasonjmcghee on Friday April 25, 2025 @06:15PM (#43799006)
I read it similarly - that this is a specific attribute of bfloat16, so the quants folks tend to run on local hardware don't have the same inefficiency to exploit

There are no comments on this comment yet.