Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Scaling LLama2-70B with Multi Nvidia/AMD GPU.
Also saw a report earlier today on MLC’s discord about AMD MI-100:
GPU Count | Model Size | Prefill Speed | Decode Speed
1 | 33b | 102.2 | 22.3
2 | 33b | 112.3 | 33.0
4 | 33b | 144.8 | 41.2
There are no comments on this comment yet.
Also saw a report earlier today on MLC’s discord about AMD MI-100:
GPU Count | Model Size | Prefill Speed | Decode Speed
1 | 33b | 102.2 | 22.3
2 | 33b | 112.3 | 33.0
4 | 33b | 144.8 | 41.2