Why DeepSeek is cheap at scale but expensive to run locally

Re:Why DeepSeek is cheap at scale but expensive to run locally by DiabloD3 on Monday June 02, 2025 @02:09AM (#44156177)

AMD is extremely bad at communications. The driver already contains everything ROCm requires to talk to the GPU, and ROCm itself is only a SDK that contains runtimes, libraries, and compilers.

This part of TinyGrad is not a driver, however it tries to hijack the process to do part of that task. You cannot boot the system with this, and it does not replace any part of the Mesa/DRI/DRM/KMS/etc stack. It does reinitialize the hardware with a different firmware, which might be why you think this is a driver.

[ Parent | on HN ]

Re:Why DeepSeek is cheap at scale but expensive to run locally by ryao on Monday June 02, 2025 @03:45AM (#44156536)

I consider it to be a driver, or at least part of one. Userspace drivers exist. Graphic drivers originally were entirely in userspace, until portions of them were moved into the kernel for kernel mode setting and DRM. These days, graphics drivers themselves have both kernel mode and user mode components. The shader compiler for example would be a user mode component.

[ Parent | on HN ]

Re:Why DeepSeek is cheap at scale but expensive to run locally by DiabloD3 on Tuesday June 03, 2025 @05:34AM (#44168081)

I'm aware. One of the biggest things in fixing the Linux desktop was no longer needing drivers in the Xserver and needing it to be suid root.

What was linked is written in Python. Nothing in Python is ever going to be a userland driver.

[ Parent | on HN ]

Re:Why DeepSeek is cheap at scale but expensive to run locally by ryao on Sunday June 08, 2025 @05:39AM (#44215840)

There is no reason people cannot write userland drivers in Python.

[ Parent | on HN ]