News for hackers. Stuff that matters.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by therealpygon on Tuesday January 28, 2025 @08:54AM (#42852297)
I honestly don’t know why people aren’t more upset by this and still get on their knees for Nvidia. They made the decision specifically to cripple consumer card memory because they didn’t like data centers were using them instead of buying their overpriced enterprise cards that were less performant. They removed NVLink because people were getting better performance out of their two $400 cards than the $1,500 cards Nvidia was trying to peddle. They willfully screw consumers and people love them for it.
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by dagaci on Tuesday January 28, 2025 @09:18AM (#42852539)
Because sensible people just use the cloud at this point, you can probably get several years of training for $6000
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by immibis on Tuesday January 28, 2025 @11:15AM (#42854039)
It buys you approximately two days (with reservation discount) of a single p5.48xlarge instance, which has 2TB of RAM, and 640GB of VRAM in 8x H100 cards. In fact that is the pricing example they use: https://aws.amazon.com/ec2/capacityblocks/pricing/
Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by dagaci on Tuesday January 28, 2025 @01:56PM (#42856377)
MI300X (RunPod) 192gb ram Hourly Rate: $2.49/hr. Break-even Point: You can rent for 2,410 hours (~100 days of non-stop-continuous use) before reaching the cost of the $6000 Mac. Mac's top out at 192GB not 2TB ;) Consideration: If your AI training requires sporadic use (e.g., a few hours daily or weekly), renting is significantly cheaper. MI300X will also get you result many times faster too, so you could probably multiply that 100 days!