Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs
by bschne on Sunday March 10, 2024 @01:13PM
(#39660751)
What makes the resulting topology great for matrix math, vs. non-matrix math workloads? Naively if you know you‘re „only“ going to multiply matrices, what do you need the flexibility and fast context-switching for? Is the end-game here that you can lay out the workload s.t. you have a series of closely colocated cores carrying out the operations of some linalg expression one after the other and the memory for intermediate results right in between, or something like that?
[ Parent
| on HN ]
Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs
by cavisne on Sunday March 10, 2024 @03:29PM
(#39661848)
I suspect thats basically it (operations one by one and then pipelining to saturate). Thats basically what Groq does also AFAIK. From their website it seems the chips are designed to be connected together into one big topology, the "Galaxy" system. Also similar to TPU's, although they use HBM with only a few very powerful "cores" vs DRAM with low powered cores.
Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs
by imtringued on Sunday March 10, 2024 @05:57PM
(#39662825)
Is this some kind of trick question?
Your core needs to be fully programmable so you can do things like kernel fusion. The simplest form is to load quantized weights and dequantize them to bfloat16 as you go. Llama.cpp and it's gguf files support various types of quantization and most of them require programmability to efficiently support them.