Curious, what trick? :)
Wishing for mainstream CPU support for anything but IEEE numbers was always a pipe dream on anything but the decades-long term, but I gotta be honest, I was hoping the current AI hype wave would bring some custom silicon for alternative float formats, Posits included.
> the addition operation takes up quite a bit more floorspace, and just quantizing to int4 is with grouped scaling factors is easier with existing gpu pipelines
Can you elaborate on this part?
Normally fp numbers have an "invisible leading one bit", so an ieee fp is (1+0.nnnn) x 2^E where nnn is the mantissa and E is the exponent.
as posits are always two's complement, you can model negative posits as having an "invisible leading two bit", aka (2-0.nnnn) x 2^E-1. this greatly simplifies circuitry for addition and multiplication.
> the addition operation takes up quite a bit more floorspace,
I miswrote, it's multiplication. This is simple: since the math requires floorspace of size n^2 where n is the maximum resolution of the mantissa, and posits have more resolution around unity.
we did floorspace estimates for fp32 versus p32 and it looked like the extra space required for the mantissa did not overtake the savings from the crazy logic required for sign flipping, nans, and denormals that IEEE requires that posits don't.
> I was hoping the current AI hype wave would bring some custom silicon for alternative float formats, Posits included
i was hoping for this too!
There's a tentative standard btw: https://posithub.org/docs/posit_standard-2.pdf
Ahh indeed! I'm working on a software posit implementation (for fun, I'll eventually post it as a rust crate) so I already make use of this trick :) decoding a p64 into an exponent and twos complement signed mantissa can be done in 9 instructions on x86!
John's background is in scientific compute/HPC and he previously advocated for using unums (which do fully track errors) and there is a version of posits (called valids) which do track errors, encouraging the user to combine with other techniques to cut the error bounds using invariants, but that requires algorithmic shift. Alas, a lot of examples were lifted from the unums book and sort of square peg/round holed into posits. you can see an example of algorithmic shift in the demo of matrix multiplication in the stanford talk (that demo is me; linked in OP).
as for me, i was much more interested in lower bit representation for ml applications where you ~don't care about error propagation. this also appears in the talk.
as it wound up, Facebook took some interest in it for AI but they nih'd it and redid the mantissa as logarithmic (which i think was a mistake).
and anyway redoing your silicon it turns out to be a pain in the ass (quires only make sense in the burn-the-existing-world perspective and are not so bad for training pipelines, where iirc kronecker product dominates), but the addition operation takes up quite a bit more floorspace, and just quantizing to int4 is with grouped scaling factors is easier with existing gpu pipelines, even custom hardware too.
fun side fact: Positron.ai, was so-named on the off chance that using posits makes sense (you can see the through line to science fiction that i was attempting to manifest when i came up with the name)