Re:TSMC execs allegedly dismissed OpenAI CEO Sam Altman as '...
by HarHarVeryFunny on Tuesday October 01, 2024 @01:58PM
(#41711862)
Sure, ANN computations are mostly multiplication (or multiply and add) - multiply an ANN input by a weight (parameter) and accumulate, parallelized into matrix multiplication which is the basic operation supported by accelerators like GPUs and TPUs.
Still, even with modern accelerators it's lot of computation, and is what drives the price per token of larger models vs smaller ones.
Drop the S, I think. There’s no time dimension.
And FLOP is a generalized capability meaning you can do any operation. Hardware optimizations for ML can deliver the same 100B computations faster and cheaper by not being completely generalized. Same way ray tracing acceleration works: it does not use the same amount of compute as ray tracing in general CPU’s.