OP here - I went with Vosk because it's pretty trivial to lock input down to a set vocabulary (in this case numbers) and from my testing works relatively well on most browsers. (webkit, firefox, etc)
this is such a poor model, I've tried it before. Try yapsnap it has better word error rate and runs super fast on CPU. https://github.com/kouhxp/yapsnap/
Thanks for the suggestion - I've been looking yadsnap over but it seems like the ONNX model for english is significantly larger than the one used by vosk (30mb versus 150mb) so that would be my only hesitation to use it.
Maybe there's a way to shrink it down given I only need to support a limited vocabulary domain...
Not familiar with the sota if someone can chime in.