Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Scalable MatMul-Free Language Modeling. Parent comment
The exp/log trick to multiply with addition does indeed look very familiar. I know that a number of demos used it in the 90ies.
Bill Dally, the Nvidia Chief Scientist also seems to be a big fan of log8 representations: https://youtu.be/gofI47kfD28?t=1953
It seems it did not make it into the hardware though, yet...
There are no comments on this comment yet.
The exp/log trick to multiply with addition does indeed look very familiar. I know that a number of demos used it in the 90ies.
Bill Dally, the Nvidia Chief Scientist also seems to be a big fan of log8 representations: https://youtu.be/gofI47kfD28?t=1953
It seems it did not make it into the hardware though, yet...