If someone come in and beats them by .001ns they get all their profits. Is .001ns really worth that reward? The utility provided is mostly detached from the reward, it is just winner gets it.

Worth what reward? Once an HFT firm buys a security, they now have to sell it again. Or vice versa if they sell a security. Do you think they earn a profit on every trade? There is risk in every trade, sometimes they overpay, sometimes they underpay.

It makes no material difference to the profits of a buy and hold investor. If it did, buy and hold investors would not trade with HFT firms.

The liquidity provider reward or index/underlying arbitrage reward, etc. Yes there is a stochastic component and various other things, I'm only talking about the edge from lower latency.

Not all markets are deterministic, in some there is some level of jitter so fastest guy doesn't always win anyway. Some markets also take steps to prevent the speed game too in how they define the rules.

While for obvious big moves it might be all about latency, most events are more about the quality of your pricing, incorporating correlated information, edge requirement and risk management that decides what you go for and what you don't, and for those smarter trades you usually have microseconds to spare.