What I've been trying to do is get it to chew through whisper.cpp at anything like a reasonable rate. It can so nearly do it with the right sized model, the latency isn't good enough yet, but the reason I'm doing this on a pi zero 2 is because of the USB-OTG port. I can take the whisper.cpp output, and have the pi pretend to be a USB keyboard, typing what it hears into a second machine over USB. This gives you dictation capabilities in environments where upgrading the PC it's plugged into isn't an option.

Like I say, it's not quite there yet in terms of usability, but it's close enough that I think the problem is just a matter of software (and arguably model) optimisation.