News for hackers. Stuff that matters.
Re:NanoGPT by imtringued on Wednesday January 11, 2023 @08:56AM (#34338471)
Here is a GPU implementation.

https://ieeexplore.ieee.org/document/9635657

It is somewhere from 8x to 25x faster than doing dense machine learning. The speedup was higher on the original CPU implementation and the GPU paper mentions that if there isn't enough shared memory on the GPU it will have to switch to an algorithm that has more overhead.

By neurons I actually meant "nodes"

My comment is effectively a summary of this article: https://www.kdnuggets.com/2020/03/deep-learning-breakthrough...

There are no comments on this comment yet.