News for hackers. Stuff that matters.
Re:4-Bit Quantization and QLoRA by llm_nerd on Thursday May 25, 2023 @09:03AM (#36070207)
This entry very casually goes from describing FP8 and then E4M3 and E5M2, to suddenly talking about FP4. It describes the demonstration of having the mantissa bits "1101" but that by itself is four bits. Unless it's including the implicit leading 1 but even then that leaves just 1 bit for the sign. So no exponent bits at all? Then later it gives an example with zero mantissa bits. ?

Are there any decent but easily digest summaries of FP4? The best I can find is a giant paper. I do not understand why the linked entry gave great summaries of the larger FP types but then waves hand / fog about FP4.

Re:4-Bit Quantization and QLoRA by synthos on Thursday May 25, 2023 @09:15AM (#36070325)
I agree that section is a bit confusing. The takeaway is that there are two 4bit formats they're using for weight compression E3M0 and E2M1.
Re:4-Bit Quantization and QLoRA by phh on Thursday May 25, 2023 @11:43AM (#36072214)
Yeah that section isn't very clear. Also, the article doesn't seem to explain what NF4/NormalFloat is at all? I would guess it has something to do with the position of the value on a gaussian, but that's just a guess