If that creates too many concerns over security of the RNG, then use deterministic rotating ranks. Traders may be able to predict who they will be able to beat out on the next tick, but the high ranks rotate throughout the herd, so everybody gets a fair shot at beating out others at the same price.
Even if you make sure to assign just one ID per trading firm, firms could coordinate to share their IDs. For example, if you're not using your ID this tick, you might place an order on my behalf for a small fee. And I'll also have sent some of my orders through an independent broker to take advantage of their ID. (Of course, if that broker places many such orders, and only some of those fill, they'll need some policy for which of their customers get the trades... maybe they could use time priority?)
It's conceivable (if unlikely) that the SEC could enact and enforce a ban on such collusion. Even then I think there would be other perverse incentives, but I'll need to think more about it.
One work-around might be to add rules to the exchange forbidding the placing of multiple bids by the same entity at the same price, and then allocating randomly.
This is also reminding me of the Talmud's descriptions of how to pay off debtors when there isn't enough money to go around: http://mindyourdecisions.com/blog/2008/06/10/how-game-theory... Such a system could work for exchanges too, although it still encourages over-bidding.
Surely also over-bidding has its own risk/reward curve so it would be reasonable to allocate pro-rata and let people work out how much to over-allocate.
* And realised what the implications were.
Elsewhere in this thread (http://news.ycombinator.com/item?id=3856015) I argue against treating orders differently based on the "entity" that placed them.
In general, we want traders to spend their time thinking about asset prices and risk, not market structure and game theory. The price-time priority system is a very simple one that rewards traders for deciding what they want and then announcing it right away. All the modifications that have been proposed in this thread encourage traders to play games, second-guess one another, or otherwise work around the system.
There are definitely tradeoffs, but personally I think it's better to use the cleaner system and accept the latency arms race than to add a layer of artificial incentives -- and for what it's worth, it appears that nearly every major electronic market has come to the same conclusion.
Finally, thanks for the link to the Talmud article: it was a very cool application of game theory to history. (Of course, I don't see any indication in the article that the Talmudic system would be an improvement over pro rata.)
So I don't think the current solution is necessarily wrong; just annoying we can't get all the benefits and also avoid the arms race.
No problem, I very much like the Talmud article and how simple the explanation works out to being.
The London Metals Exchange - being an outlier as always - has gone for a different approach: they throttle traffic to 40 actions/client/second and smooth it out over as many seconds as necessary - so a 10,000 quote burst will be smeared over 4 minutes... good luck trying to manipulate the market with that, HFT boys.
Every market has a book, and a price. If a sell comes in at under the price, and there are unfulfilled buy orders anywhere above the price, the sell executes at that price for the order that has been above that price the longest. Conversely if a buy comes in above the price and there are unfulfilled sell orders below that price, the trade executes at that price for the sell order that has been below that price.
The price itself drifts upwards or downwards at a set rate depending on whether there are currently an excess of buy or sell orders that would execute at the current price.
What downside do you see in such a scheme?
This would result in much smaller incentive for speed and less money and energy spent on being faster, and more fair markets, etc.
And traders already have the option of advancing their price by a penny. The issue is that they are competing over hundredths of a penny.
http://www.tse.or.jp/english/faq/list/stockprice/p_b.html
But yeah, it wouldn't be a good method for an every-N-seconds auction.
Let's say we discretize the exchange to, say, 1 Hz, so all the orders are queued up and then executed simultaneously at the next clock tick. Now, on a given tick, there will almost always be a mismatch between the number of buy orders and the number of sell orders, so some orders will go unfilled. How do we choose which ones? (For this discussion I am ignoring price. Obviously we will fill more aggressively-priced orders first; the question is how to prioritize orders of equal price.)
We could give priority to the ones that arrived first, but of course then we're back to traders racing each other. The only other scheme I can think of [1] is to fill every order in proportion to its size. So if Alice wants to sell 200 shares, Bob wants to buy 100 shares, and Charlie wants to buy 300 shares, we give 50 shares to Bob and 150 shares to Charlie.
Well, this scheme has its own problems -- arguably worse ones than the current system! If Bob really wants to buy 100 shares, and expects to be competing with Charlie, he has an incentive to place a much bigger order: in this case, if he knew Charlie's bid size, he would also ask for 300 shares, expecting to get 100. Of course, Charlie will be playing the same game and inflating his own bid size. The equilibrium is that everyone asks for way more size than they actually want.
There are two things wrong here:
1. Traders are spending mental energy trying second-guess each other's order size; those who don't are crowded out of the most competitive (and, typically, profitable) trades.
2. Since traders can't guess correctly every time, they will sometimes end up buying or selling much more than they want, which means they have to turn around and do the opposite trade, paying the spread in the process. If this happens during during a large price movement, these traders can lose even more money, and in their haste to reverse their trades, they will drive the market even further in the same direction.
And by the way, this is not just speculation. While I'm not familiar with any exchanges that discretize their clocks this way, there is a product with a similar system: the CME Eurodollar contract [2]. In this contract, unlike the other futures products traded on the CME, resting orders do not execute on a first-come-first-served basis; instead they are allocated "pro rata", or in proportion to their size, much as I described above. Based on my conversations with several people who trade them (including HFTers), the results are also as I described: traders routinely over-order and suffer the consequences.
Hopefully this sheds some light on why this seemingly obvious solution is not widely implemented.
[1] Of course I can think of many other schemes, but they either have obvious problems or basically reduce to this one.
[2] There are other products matched this way, but I'm not as familiar with them.