Not all markets are deterministic, in some there is some level of jitter so fastest guy doesn't always win anyway. Some markets also take steps to prevent the speed game too in how they define the rules.

While for obvious big moves it might be all about latency, most events are more about the quality of your pricing, incorporating correlated information, edge requirement and risk management that decides what you go for and what you don't, and for those smarter trades you usually have microseconds to spare.