IANAL, but I think that's only true in the USA. Most other places are first to file, not first to use.

And even then, that doesn't prevent a frivolous suit in the USA

No, for trademarks it's standard practice to allow prior use to continue (USA, EU, UK, at least). You can have a trade mark for decades, and be protected by passing-off law, then come to register it and find a brand new company has registered that mark, but that doesn't mean you can't continue your own trade with that mark, but can mean you're going to infringe on their registered right if you change (eg enter a new TM class, enter a new geographic region).

This prevents established companies from being damaged by opponents registering their marks.