News for hackers. Stuff that matters.
Re:DeepSeek 4 Flash local inference engine for Metal by LoganDark on Thursday May 07, 2026 @11:50PM (#48058325)
When you support multiple backends, you end up having to abstract over them. Each backend may implement the abstraction to the best of its capability, but you still have to deal with the abstraction sitting between your workload and its compute. Wouldn't it be nice if you didn't need that abstraction? That's what GP is talking about, I'm sure: optimizing the workload directly for the hardware, rather than merely the workload and the backend for the abstraction.
Re:DeepSeek 4 Flash local inference engine for Metal by Muromec on Friday May 08, 2026 @09:46AM (#48063079)
Absttaction doesnt always imply performance overhead.
Re:DeepSeek 4 Flash local inference engine for Metal by LoganDark on Friday May 08, 2026 @03:54PM (#48067897)
Abstraction necessarily reduces fit to the hardware when multiple different kinds of hardware are supported. Whether that is towards the hardware you are using varies, but in many cases it is, which means you can reach performance gains by shedding the additional support to focus on just your hardware.