Depends how much you care about people talking over one another. If your call is a presentation/lecture/class with few switches between speakers, latency's no problem.

But if your calls normally have lively discussion where someone different jumps in any time there's a pause, the higher the latency the more likely people will say "meeting in person is much better"

Likewise, with things like remote desktop, 100ms of latency isn't a dealbreaker but it'll certainly leave some of your users saying "things that run locally just feel snappier"