I've been working on a browser API that lets websites run specific on-device open-weight models, with the browser handling inference and permissions. I think giving developers the ability to pick the right tool for the job is critical. This shouldn't require every site shipping their own inference runtime, libraries and models.

I'm working on a working implementation as a proof of concept (I'll share soon). Still a lot to figure out. Would love to know what people think.