Thank you. These keywords (Kaldi, Dragonfly) put me on the right track and I found nerd-dictation [1] which is easy to use and hackable. Perhaps not useful for your programming purposes, but I need something for writing prose and it's a decent start for that.

[1] https://github.com/ideasman42/nerd-dictation