It is enough to run the dynamically quantised 1.56 bit version I believe, which is fun to play around with.
Re:Deepseek R1-0528
by zackangelo on Thursday May 29, 2025 @02:37AM
(#44123635)
Yeah, to run the full precision model you need either two 8xH100 nodes connected via Infiniband or one 8xH200 node or one 8xB200 node.
Not for the GPU poor, to be sure.
An alternative is to use serverless GPU or LLM providers which abstract some of this for you, albeit at a higher cost and slow starts when you first use your model for some time.