Re:Transformer Inference Arithmetic
by thatcherc on Monday May 15, 2023 @12:24PM
(#35950730)
Author seems to be using billion == 10^12 instead of the common billion == 10^9. A lot of the math still works out since there's a multiply and a divide by a billion, but it is a little confusing to see passages like this:
> Given the parameter count, we can multiply by two to get bytes. So to calculate the size of the weights for a 52B model.
Re:Transformer Inference Arithmetic
by pulvinar on Monday May 15, 2023 @12:39PM
(#35950956)
I assume that's a slip back to the old British billion. But luckily that mistake doesn't affect that result, and does get it right in the next equation:
> Given the parameter count, we can multiply by two to get bytes. So to calculate the size of the weights for a 52B model.
> 52e12⋅2 = 104e12 bytes ≈ 104GB