News for hackers. Stuff that matters.
Re:Deepseek R1-0528 by omneity on Wednesday May 28, 2025 @06:23PM (#44121188)
Worth mentioning that a single H100 (80-96GB) is not enough to run R1. You're looking at 6-8 GPUs on the lower end, and factor in the setup and download time.

An alternative is to use serverless GPU or LLM providers which abstract some of this for you, albeit at a higher cost and slow starts when you first use your model for some time.

Re:Deepseek R1-0528 by girvo on Wednesday May 28, 2025 @08:25PM (#44121845)
It is enough to run the dynamically quantised 1.56 bit version I believe, which is fun to play around with.
Re:Deepseek R1-0528 by zackangelo on Thursday May 29, 2025 @02:37AM (#44123635)
Yeah, to run the full precision model you need either two 8xH100 nodes connected via Infiniband or one 8xH200 node or one 8xB200 node.

Not for the GPU poor, to be sure.