Tenstorrent unveils Grayskull, its RISC-V answer to GPUs

Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs by andy99 on Sunday March 10, 2024 @11:03AM (#39659717)

  This is purely a developer kit, letting people get used to the hardware configuration and the software stacks before the big hardware comes later in future generations.

I'd like to know what different precisions/ quantizations if any are supported. For LLMs 8GB is fine for playing around if weights are quantized. And as the article mentions it's more than enough for lots of computer vision models.

[ Parent | on HN ]

Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs by camel-cdr on Sunday March 10, 2024 @11:17AM (#39659834)

Grayskull supports FP8, FP16, BFP16, VFP2, BFP4, BFP8 [0], and some sort of 19-bit floating point in the SFPU. [1]

I don't know how and with what performance those are supported.

FP4 has 221 TFLOPS on the Grayskull e75, and 332 TFLOPS on the Grayskull e150. [0]

While wormhole has 82 INT8 TOPS per card. [2]

I haven't found any other numbers.

Edit: Some more info about data formats: https://docs.tenstorrent.com/tenstorrent/v/tt-buda/dataforma...

[0] https://docs.tenstorrent.com/tenstorrent/add-in-boards-and-c...

[1] https://tenstorrent-metal.github.io/tt-metal/latest/tt_metal...

[2] https://tenstorrent.com/systems/galaxy/

[ Parent | on HN ]

Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs by andy99 on Sunday March 10, 2024 @04:45PM (#39662381)

Cool, thank you!

[ Parent | on HN ]

Re:Tenstorrent unveils Grayskull, its RISC-V answer to GPUs by alecco on Monday March 11, 2024 @03:49AM (#39665697)

That's a lot of formats for FP8, FP4, and even FP2! (never heard of). I wonder how they did it. Nice.

[ Parent | on HN ]