If using chatGPT advance voice mode, the recent upgraded version seems better, plus, if you're on an iPhone, you can turn on Voice Isolation in the Control Center which will filter out almost all sounds from the phone microphone except for your speaking voice, which made chatGPT behave as one would hope -- I believe the setting is specific to the current microphone using app.