Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs
by buildbot on Sunday March 10, 2024 @02:00PM
(#39661114)
Wormhole does support scale out though via it's built in networking? And there seem to be links for something on top of the board, NVLINK style.
And yes, I know, I've been working on LLM pretraining for about 4 years now, since 2020. The number formats themselves so far are mostly scale invariant or improve with larger scale - you can quantize a larger model and see less performance drop than a smaller model.
And yes, I know, I've been working on LLM pretraining for about 4 years now, since 2020. The number formats themselves so far are mostly scale invariant or improve with larger scale - you can quantize a larger model and see less performance drop than a smaller model.