As I mention in another comment, I actually do now use Whisper. In fact, I just spent the last few hours using Whisper and GPT to write some drafts.

Whisper has a very serious downside: It often ignores quite a bit of relevant audio, and I dealt with this many times in the last few hours, to the point where I'm considering finding an alternative.

See this report, for example:

https://community.openai.com/t/whisper-leaves-out-chunks-of-...