jimsrand
VAD, ASR, a small language model, and TTS all run locally on iOS, with a cross-platform architecture that can swap the local model for a frontier cloud LLM when needed.
For now, it's just a button that lets you switch between the on-device SLM and the cloud-based LLM. But we're planning to implement different routing options (e.g. automatic fallback when internet is available, or confidence based, etc.).
Would love to see this remixed. It's designed to be easy to swap components (e.g. use a different VAD, STT, TTS, different language models, etc.).
The implementation is open source. You'll see there's a dependency on the Switchboard SDK. Currently, the free tier is only to 10K MAUs, but we're planning to remove that limit and make most of the core offering a totally free service -- so get in touch if the current model is a blocker for some reason.
Show HN: EdgeSpeech, on-device speech-to-speech for React Native
At some point I suspect you'll want offline functionality / on-device for some aspects of it. Would be interested to chat if you do. Look us up at switchboard.audio -- we are now building dev tools for applications like yours.