Re:Triton: Open-Source GPU Programming for Neural Networks
by enos_feedler on Thursday August 05, 2021 @11:58PM
(#28083183)
differentiation isn't really the key algorithm. In fact, I believe that happens at compile time, not run-time. So not only is not key, it isn't even happening at run-time.
For example, in neural networks the key algorithm is differentiation. That is implemented the same way on both CPUs and GPUs.