DeepSeek-R1 with Dynamic 1.58-bit Quantization

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by karamanolev on Tuesday January 28, 2025 @06:10AM (#42851076)

For personal usage, does it matter though? In most places residential electricity is cheap compared to everything else. In a DC context I feel it matters a lot more compared to the capex.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by thenickdude on Tuesday January 28, 2025 @06:45AM (#42851297)

1x 3090 (350W power limit) already makes it feel like I'm running a fan heater under my desk, 5x would be nuts.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by heelix on Tuesday January 28, 2025 @08:01AM (#42851806)

I think the last time any of my computers had a case was back when I realized the pair of 900gx2 cards I was running was turning my computer into an easy bake.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by TeMPOraL on Tuesday January 28, 2025 @07:40AM (#42851656)

Place and time your use right, and you'll save a bit on heating at winter and/or at nights.

[ Parent | on HN ]

Re:DeepSeek-R1 with Dynamic 1.58-bit Quantization by ryao on Tuesday January 28, 2025 @03:47PM (#42857758)

When running inference workloads via something like llama.cpp, only 1 GPU is ever used at a time, so you would have 1 active GPU and 4 idle GPUs. That should make the power usage less insane in practice than you expect.

[ Parent | on HN ]