DeepSeek-R1 with Dynamic 1.58-bit Quantization

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by jmb99 on Tuesday January 28, 2025 @10:39AM (#42853551)

It’s more expensive, but Zen4 Threadripper Pro is probably the way to go on that front. 8 memory channels, with DIMMs available up to DDR5-7200 for 8x32GB (256GB), or DDR5-6800 for 8x48GB (384GB). It’ll set you back ~$3k for the RAM and ~$6k for a CPU with 8 CCDs (the 7985WX, at least), and then ~$1k for motherboard and however much you want to spend on NVME. Basically ~$10k for a 384GB DDR5 system with ~435GB/s actual bandwidth. Not quite as fast as the 192GB Apple machines, but twice as much memory and more compute for “only” a few thousand more.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by sourcecodeplz on Tuesday January 28, 2025 @12:20PM (#42855018)

At these prices, I would just get 2xDigits for $6k and have 256gb.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by riku_iki on Tuesday January 28, 2025 @04:17PM (#42858142)

is it confirmed that you can get 256gb of vram for that amount? Because my understanding is that digits pricing will start at $3k for some basic config.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by fluoridation on Tuesday January 28, 2025 @04:40PM (#42858411)

What they meant is buying two whole separate computers.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by riku_iki on Tuesday January 28, 2025 @04:51PM (#42858530)

I understand. It is still unclear if you can get 128GB vram for $3k.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by fluoridation on Tuesday January 28, 2025 @05:01PM (#42858636)

Well, I mean, the press release is pretty unambiguous.

>Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage.

Even if $3k is only the starting price, it doesn't sound like spending more buys you more memory.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by riku_iki on Tuesday January 28, 2025 @05:19PM (#42858848)

Ok, but it is not clear what kind of RAM is that, how many memory channels, etc. If the goal is to have just 128GB of some ram, then it could be achieved by paying few $100.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by fluoridation on Tuesday January 28, 2025 @05:27PM (#42858934)

Fine, but at that point you're arguing about the concept of the product. It's billed as a computer for AI and you're saying that it might not be more suitable for AI than a regular PC.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by riku_iki on Tuesday January 28, 2025 @05:30PM (#42858972)

it is possible that one could build better PC than digits for AI. We will see once they release digits.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by jadbox on Tuesday January 28, 2025 @03:11PM (#42857338)

I have a feeling that Digits will probably get sold out and will pricing will get hiked WAY up.

[ Parent | on HN ]