Re:Local AI needs to be the norm
by diwank on Sunday May 10, 2026 @11:11PM
(#48090676)
in order for us to get there, i think we need a standardized api at the os layer for local models so that the os could optimize, batch and safely allocate resources. something like an analog of chrome's local model "prompt" api but provided and managed by the os itself. the user can choose which model they want to primarily use and so on but all of the heavy lifting and continuous batching is done automatically by the os
[ Parent
| on HN ]
There are no comments on this comment yet.