100ms isn't that low, and those NATs are introducing additional latency and complexity that don't need to exist. If both sides are behind NAT then you may well need to relay the connection through a third party server, introducing even more cost, complexity and latency and involving an otherwise-unnecessary third party.
"I managed to get something working" doesn't invalidate any of that.
Network latency added by natting is microseconds.
Imagine I had a public IP - no natting involved. How would my video client tell my stateful firewall to allow a new incoming connection from the calling device?
Once you have to bounce the data through a hosted proxy, you're forcing no less than 10 ms on to the users (of course there are rare exceptions for users really close to your proxy).
NAT is a thing that creates real problems, for example inhibiting low-latency video calls.