News for hackers. Stuff that matters.
Re:Llama 3.1 Official Launch by bick_nyers on Tuesday July 23, 2024 @02:59PM (#41049545)
The best NVLINK you can reasonably purchase is for the 3090, which is capped somewhere around 100 Gbit/s. This is too slow. The 3090 has about 1 TB/s memory bandwidth, and the 4090 is even faster, and the 5090 will be even faster.

PCIE 5.0 x16 is 500 Gbit/s if I'm not mistaken, so using RAM is more viable an alternative in this case.

Edit: 3090 has 1 TB/s, not terabits

There are no comments on this comment yet.