Suppose we had no weather forecasts and meteorologists hoarded the information. And suppose store owners could only adjust prices each morning. Before a hurricane, meteorologists could buy up every last emergency supply within an afternoon from the stores and then sell them the next day after the hurricane at 10 times the original price. Is that fair?
For your meteorology example, if such information didn't have life or death consequences, I might call it fair. But the contexts of the two scenarios are different. The stock market is inherently a bet on future prices, and so using information obtained by yourself through your own efforts cannot be seen as unfair--it is an expected property of the system. Using HFT techniques to profit off of frontrunning your demand is unfair. The "market" for items needed for survival isn't analogous.
On the other hand, I can imagine there being a trivial limit as far as the wider economy is concerned, but still eliminating the benefit that colocation gives HFT firms. Do you disagree?
They think that HFT makes them materially worse off when it, in fact, does the exact opposite. We should just cave to that misconception?
Misconceptions should certainly be eliminated, but the fact that HFT firms require colocation does not pass the smell test of fairness. No amount of explanations can eliminate that.