Auction theory is all about achieving that for simple, single round auctions. Why not redesign the stock market in a similar way as well? I suppose it could make for a nice PhD thesis (or several).
I don't see why this is so. If I have decided through some analysis that I am going to sell X shares at some date, why shouldn't that information be private? Front-running my behavior is essentially front-running my analysis. It would be unethical to hack my computer and read my analysis for yourself, so how is analyzing my behavior to get at that same information any different?
Why is the market entitled to knowledge that I'm currently in the process of making a big order? This knowledge being disseminated clearly doesn't benefit me. Should I not be entitled to the sole benefit of my reasoning in deciding to make a big order? Furthermore, how does this knowledge being disseminated mid-order rather than after the full order has completed benefit the market as a whole?
Your knowledge that you're placing an order of size X, and why you're placing an order of size X, is always going to be more precise than what's gleaned from observing your behavior, and thus you'll still have an advantage.
"Furthermore, how does this knowledge being disseminated mid-order rather than after the full order has completed benefit the market as a whole?"
For a large enough order in a sufficiently illiquid market, "after the full order has completed" could be days later. More things being priced in means more accurate prices, and that seems to be of some benefit to "the market as a whole." Though to be sure, the longer time frame isn't terribly relevant to the "predatory traders" across exchanges that Chris had been discussing.
Edited to add: Also, the market doesn't learn anything about you in particular - orders are anonymous. What we learn directly is "a trade happened for X lots at $Y", along with possibly which side was the aggressor (depending on the exchange).
Suppose we had no weather forecasts and meteorologists hoarded the information. And suppose store owners could only adjust prices each morning. Before a hurricane, meteorologists could buy up every last emergency supply within an afternoon from the stores and then sell them the next day after the hurricane at 10 times the original price. Is that fair?
For your meteorology example, if such information didn't have life or death consequences, I might call it fair. But the contexts of the two scenarios are different. The stock market is inherently a bet on future prices, and so using information obtained by yourself through your own efforts cannot be seen as unfair--it is an expected property of the system. Using HFT techniques to profit off of frontrunning your demand is unfair. The "market" for items needed for survival isn't analogous.
On the other hand, I can imagine there being a trivial limit as far as the wider economy is concerned, but still eliminating the benefit that colocation gives HFT firms. Do you disagree?
They think that HFT makes them materially worse off when it, in fact, does the exact opposite. We should just cave to that misconception?
Misconceptions should certainly be eliminated, but the fact that HFT firms require colocation does not pass the smell test of fairness. No amount of explanations can eliminate that.
Let me see if I understand your example. I want to dump 100k@100$ and yet the current book doesn't have that many buy orders (otherwise the order would just complete). The book has buy 1k@100 and sell 1k@105 So I try to be sneaky and just clean out the book with 1k@100, selling 100 shares each to 10 retail investors and wait for more 100$ retail investors to show up by placing another 1k@100 sell order. The HFT sees this quickly and puts in orders for buy at 90$ and sell at 95$. Now the next retail investor sees this and just buys at 95 instead of 100, saving 5. I need to then sell at 90 instead of 100, losing 10. The HFT firm makes 5.
So the HFT firm is taking money from informed investors and taking some for itself and giving some to people who have no special information about the market like retail investors. If that is the case the only problem with HFTs are second order. That (much like bitcoin) they spend a bunch of resources with busywork that could be at least partially eliminated by more decimal points in pricing trades. And that they may discourage some informed investors from adding information to the market because they can't profit from it, reversing their benefit.
It’s perfectly reasonable to argue that shouldn’t be allowed, but then you are (implicitly) arguing that either (a) complete market information should not be publicly available or (b) there should be some mechanism which prevents some individuals from acting on that information as quickly as they would like to.
If I want to sell 100,000 shares of Yelp (which is a multi-million dollar transaction), the fact that several million dollars of Yelp is about to hit the market is a really, really important bit of information. I have a reason to hide that from the rest of the market at least until I can complete my sale. HFTs profit by ascertaining my intentions faster than I can complete my intended action and then broadcasting my intentions to other market participants via the ticker tape, whereas previously my intentions were proprietary information locked in my head.
An argument against HFT is, necessarily, an argument that I should be allowed to benefit from material non-public information about the near-term state of trading in Yelp.