Interesting, thanks for the link. This would be much fancier than the simple thresholding I'm doing currently.
It might be overkill for the use cases I'm targeting right now (clean recordings, probably done with audio gear) but I'll make a note to check it out.
https://github.com/xiph/rnnoise