I'm making LLMs work on small and old devices

Re:I'm making LLMs work on small and old devices by quickthoughts on Friday November 13, 2026 @09:22AM (#50483914)

I've also been focusing on ultra efficient local ASR models (for real time transcription) that can run comfortably on a web browser on low end phones/devices. it's actually crazy how capable it is already (https://starbright.quickthoughts.ca/) WER is like ~5 and it works on old phones that I've tested.

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by mrashad74 on Friday November 13, 2026 @09:31AM (#50484059)

looks nice! what languages does it support?

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by quickthoughts on Friday November 13, 2026 @09:33AM (#50484094)

rn just english but I have a welsh (cy) asr model cooking as well, though its still unusable (~25 WER). these tiny models are actually really cool example of distilling but also really really focusing on the engine in a way that takes advantage of crazy compression, decoding etc strategies

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by iceflinger on Friday November 13, 2026 @10:29AM (#50484957)

I've been interested in ASR models specifically for transcribing low-bandwidth low-quality radio recordings, are there any specifically focused on this angle in particular around?

[ Parent | on HN ]