Main downsides I knw of are a slightly worse codec, and in webrtc mode bandwidth needs to scale with participants so for large meetings you need to pay for hosting somehow.

Neither are reasons not to use it as a default first option.

Also matrix has voice and video now and there's big blue button.