News for hackers. Stuff that matters.
Re:The stock market is controlled by algorithms that are fig... by btilly on Thursday May 12, 2011 @05:18PM (#2542356)
Something that I've never understood.

We have markets that are based on allowing trades to happen at any agreed on price, as fast as people's minds can meet. Which can therefore result in very high volatility.

Why hasn't anyone tried building a market that goes the opposite way? Prices drift at a fixed rate (eg min(1 cent, 0.05% of price)/minute) in a direction determined by whether there is an excess of buy or sell orders, and trades execute as soon as both a buy and a sell are available at the right price. This would give greatly reduced instant volatility in return for uncertainty about how quickly orders will complete.

The advantage of the latter strategy being that it is very resistant to attempts to compromise it via high frequency trading techniques. That 1/100th of a second time margin doesn't matter because the price changes too slowly. Therefore a long-term investor knows that they aren't getting charged "rent" via the automated algorithms.

It might be objected that lack of trading volume would make such a market initially useless. But arbitrage would keep its price close to the mainstream market. Hmmm..I guess it initially comes down to whether you're paying more in rent on arbitrage or on HFT...

Re:The stock market is controlled by algorithms that are fig... by hristov on Thursday May 12, 2011 @06:44PM (#2542656)
For markets to work one must try to get as much trading as possible on them. Only if the majority of trading happens on markets will markets be able to effectively discover prices.

If one puts such artificial limitations on markets, than there is always a danger that people will not trade there. If people trade in side deals instead of the markets, then markets would not be able to discover prices and all kinds of inefficiencies would result from the uncertain prices. For example if you put in your limitation, there will be occasions where both buyers and sellers will want to buy and sell at a certain price but the exchange will simply not let them. In this case the buyers and sellers will try to meet and do the transaction outside of the exchange and that is usually a bad thing.

I think there are other ways this can be fixed. For example one may even the playing field by simply adding a clock to the market so that all trades must happen on clock "ticks". This would prevent the advantage of HFT. Or the exchanges could just stop selling offer data to the HFT people. This would not cause the market problems discussed above, because it would only remove things that actual market participants do not have anyways. Thus, these measures would not remove any actual supply or demand from the market.

The solutions are not hard to figure, the problem is that exchanges do not want to implement them. Exchanges make a lot of extra money from HFT and want to keep and grow that money even if endangers uncertainty in their markets.

Re:The stock market is controlled by algorithms that are fig... by yummyfajitas on Friday May 13, 2011 @08:10AM (#2544332)
For example one may even the playing field by simply adding a clock to the market so that all trades must happen on clock "ticks". This would prevent the advantage of HFT.

No it wouldn't. At 12:00:00:001, all the HFT's will race to place an order and be the first in the queue. At 12:00:01:000, the fastest HFTs will be the ones to trade.

Additionally, if news occurs at 12:00:00:998, the HFTs will all race to cancel their orders as fast as possible.

Re:The stock market is controlled by algorithms that are fig... by yummyfajitas on Thursday May 12, 2011 @05:41PM (#2542442)
What is the advantage of such a market? Near as I can tell, what would happen is that whenever the market believes the price should move, liquidity will drop to 0. Eventually the market will allow trades at the right price, and liquidity will return. Why is this a good thing?

Also, HFTs will still exist. Suppose the "right price" is $50 - even if the market hasn't adjusted (say it's stuck at $51), HFTs will race to have their orders first in the queue at $50 for when the market does readjust.

Re:The stock market is controlled by algorithms that are fig... by joshu on Thursday May 12, 2011 @09:37PM (#2543147)
Continuous auctions have the advantage of being continuous at the expense of volatility.

You could instead have a single auction every day and this would be more fair but less available.

Your example is not unlike how specialists control the price for markets.

You should probably look into how these things actually work.

Re:The stock market is controlled by algorithms that are fig... by dpapathanasiou on Thursday May 12, 2011 @05:27PM (#2542393)
There's been a lot of academic research on call auctions (e.g. http://www.springer.com/business+%26+management/finance/book...) which are somewhat similar what you describe in terms of their effects on price stability (though pricing happens differently), but they've not been successful in practice.
Re:The stock market is controlled by algorithms that are fig... by nradov on Thursday May 12, 2011 @07:19PM (#2542747)
HFTs aren't charging us long-term investors any "rent". They're actually giving us better trade execution. Some HFT algorithms might temporarily distort the market price of certain securities but over the long-term those minor inefficiencies get arbitraged away.
Re:The stock market is controlled by algorithms that are fig... by btilly on Thursday May 12, 2011 @08:01PM (#2542878)
Short term trading is a zero sum game. Therefore the profit that HFTs make is entirely money that in their absence would have been captured by other investors. Given that HFTs consistently do make profit, that is a form of rent charged on other players in the market.
Re:The stock market is controlled by algorithms that are fig... by vecter on Thursday May 12, 2011 @09:13PM (#2543085)
The market is a place for risk transfer. That is where the profit that HFT makes comes from. Long term investors transfer the risk of being or not being in a position to HFT market makers, who then (in expectation) make a small profit per transaction.

The bigger picture that your argument misses is that trading is zero sum in wealth, but NOT zero sum in utility. That is the amazing thing about trading (and by trading, I mean trading in general). Say I have three oranges and you have two bananas. It turns out that I really really love bananas, and you have equal preferences for oranges and bananas. I would be happy to exchange two of my oranges for one of your bananas. The next result is an increase in both of our individual utilities. That is how trading work (and why people trade), and HFT is no different.

Re:The stock market is controlled by algorithms that are fig... by cma on Thursday May 12, 2011 @10:31PM (#2543305)
Here's a better analogy:

Bananas and oranges go by nightfall after arrival in the day by ship, and there are no other sources of bananas and oranges. For every banana a person eats, they must also eat an orange.

Ships arrive sporadically throughout the day, and a ship can only contain bananas or oranges, not some of each. Only a few ships arrive each day. An equal number of bananas an oranges are shipped each day from the sources, but some of the ships sink on the way, so that there may be a surplus or deficit of bananas or oranges (remember, to eat one banana you must eat one orange).

Spotters see the ships coming at roughly the same time, with some randomly variation, and immediately adjust the price of existing bananas or oranges based on what the incoming ship is carrying--after all it didn't sink, and the fact that it didn't gives some information on whether or not there will be a banana/orange deficit.

One day a man invents a telescope and begins seeing the arriving ships five minutes before everyone else. He provides no net-wealth to society as a whole, but with the money he makes via arbitrage he begins commanding a disproportionate share of society's wealth for his own use.

*Galileo originally used his telescope invention for exactly this (in conjunction with a wealthy family)--one of the least useful things he ever did. Would you rather someone work in HFT (scope out banana boats 5 minutes earlier than the next guy) or work in something useful like scientific research/general technological improvement (Galileo's discovering of deeper secrets of the heavens/improvement of navigation and cartography enabled by the scope)?

Re:The stock market is controlled by algorithms that are fig... by forensic on Friday May 13, 2011 @12:36AM (#2543558)
Is that 5 minutes of advanced information truly worthless?

Does the average wealth of society increase or decrease with the addition of the arbitraging Galileo?

I think it increases.

Inequality increases, true, and there are other dangers in inequality.

But the median wealth of everyone also increases because now people know 5 minutes earlier if they're going to eat oranges tonight. Presumably that information is worth something, and I think there is a good argument to be made that the value of that information is equivalent to the profit of all the arbitrageurs.

I believe in wealth redistribution, but this question is less about inequality and more about the utility of that advance information.

Intuitively it does seem ridiculous that 3ms of ping is worth $300,000,000. But we're dealing in a global economy with nearing 7 billion people, and 3ms * 7,000,000,000 adds up to 8 months of human life.

It would be a problem if the fiber line owner, with his wealth, gains some kind of hegemony over the world and enforces some kind of human-life-crushing totalitarianism. But the inequality argument is most sensibly solved by wealth redistribution - taxes, welfare, public healthcare, etc. Rather than outlawing arbitrage.

Re:The stock market is controlled by algorithms that are fig... by cma on Friday May 13, 2011 @01:19AM (#2543624)
> Is that 5 minutes of advanced information truly worthless?

>But the median wealth of everyone also increases because now people know 5 minutes earlier if they're going to eat oranges tonight.

Since we can assign an arbitrary utility value to that knowledge while not changing the profits telescope man makes (how much is the existing stock of oranges and separately the existing stock of bananas now worth, now that telescope man sees a boat of bananas has arrived unsunk and each banana must match an orange? He can use this to revalue the two and sell or buy from/to people who only know the old valuation).

The utility you claim isn't directly related to the telescope-man profit. Certainly not equivalent as you seem to think:

>I think there is a good argument to be made that the value of that information is equivalent to the profit of all the arbitrageurs.

What makes you think there is a one-to-one correspondence? Can you make the argument, or do you just think there is a good one to be made? If you already made the argument I didn't see it, and if it involved multiplying ping delta by current world-population then I am definitely not clever enough to understand it. ;)

("go by nightfall" in my original post should have been worded "go bad by nightfall")

Re:The stock market is controlled by algorithms that are fig... by forensic on Friday May 13, 2011 @02:02AM (#2543727)
My point regarding pings was that tiny advantages in speed, if leveraged across enough people, become large advantages in speed, and if a single person provides a tiny bit of value to many people, then that adds up to a lot of total value produced by one person and therefore a big paycheck for that one person.

The telescope information is valuable to fruit sellers and fruit consumers alike. If Bill is very hungry today, but Joe is not, then Bill is rightly willing to pay a higher price for fruit.

Let's say that the loss of one ship raises the fruit price from $1 to $10. Bill wants to eat today and so he is saving his last $10 for when the fruit arrives.

As the ship crosses the ocean, the closer it gets to shore, the less risk there is in Bill spending his other $9. He wants to buy beer but is unwilling due to the risk of not being able to afford fruit.

To Bill, there is clear value in knowing how far along the ship has gone. The sooner he knows the better. Since knowing the ship is coming frees up his $9, this information also provides value to the brewer who will be able to sell his beer sooner. If the brewer sells his beer sooner, the brewer will be able to buy himself new shoes sooner, and so on. The 5 minutes of foreknowledge results more economic activity, and higher wealth in general. Bill gets his beer sooner, making him happier sooner. This has value. Mr. Guinness the Brewmaster gets his shoes sooner, easing his ankles sooner, and this is worth something. Every little bit of foreknowledge helps.

All arbitrage follows this same model. Whether the arbitrage profits are collected by one person or many is not relevant. There is utility in prices that accurately reflect supply and demand. The sooner prices reflect reality, the better.

If there are going to be fewer oranges available, the price should be higher to ensure an efficient distribution - a distribution to those who most value the oranges.

Galileo made prices 5 minutes more accurate, and while that is unsexy, it saved 5 minutes of inefficient trading. In aggregate, those 5 minutes end up being worth a lot.

Those minutes provide some level of utility equal to or greater than Galileo's profit. If Bill did not get at least $10 worth of utility from his oranges, he would not buy them, and Galileo would not receive Bill's money. These are voluntary transactions after all. Bill can always starve until tomorrow, or boil his shoes, or go forage for berries.

Fast information is worth something in itself.

Re:The stock market is controlled by algorithms that are fig... by cma on Friday May 13, 2011 @02:43AM (#2543838)
First you were arguing that the profit equaled the utility exactly. Now you are saying something different. Taking your new example, since the trading profits are roughly the same whether the advantage is 5 minutes or 20 seconds so long as the ship is seen first by telescope man, yet the probability of Bill mistakenly wasting his money on beer when he should have saved for bananas does depend on that length of time, then the utility and the profit aren't equal; and as the time advantage approaches 0, (and thus the utility you are talking about here approaches 0) the profit remains the same.

You are saying there is always some utility. Ok fine, but it gets arbitrarily close to 0. In the case of HFT there is also some cost (brain-drain for proprietary R&D, power drain for datacenters, etc.). Which outweighs which? What timescale do we have to get to? We are already well below it.

Re:The stock market is controlled by algorithms that are fig... by forensic on Friday May 13, 2011 @05:12AM (#2544067)
I was wrong saying profit equaled the utility exactly.

There is clearly an opportunity cost with HFTs, and I would hypothesize that at a certain point there is a better ROI in other fields for those people. There should be diminishing returns to investments in HFT, right?

This is such a theoretical argument and my conclusions are based mainly on the idea that if there is no utility people would not pay for it. It does seem ridiculous that moving 3ms faster wins you so much, but I'm just not convinced that there is definitely no utility there. Often economic utility is very hard to intuitively recognize -- often it is not recognized until people start intervening in the market and bad things start happening. If you ask computer scientists which jobs are useless, provide no utility, and should be eliminated, they would eliminate huge sections of the economy.

Re:The stock market is controlled by algorithms that are fig... by vecter on Thursday May 12, 2011 @10:48PM (#2543346)
Your analogy is pretty good, but the part about the telescope is critically flawed. You assume that there is very little (zero?) variance in the future once the man with the telescope sees the incoming ship. In the scenario you described, that may be the case.

Unfortunately, in the stock market, there is no where near that level of certainty about the future. There is great risk to posting prices that may in fact be terribly wrong in the future. That is the risk that market makers have to grapple with and quantify when they post or take liquidity.

Suppose that every time you spotted a ship through the telescope, it would magically explode 2 minutes later. This happened roughly 50% of the time, as observed over 1 million ship sightings for the past few years. This is a lot closer to what the stock market is like. The other difference is that every serious player in the stock market also has a telescope :(

Re:The stock market is controlled by algorithms that are fig... by cma on Thursday May 12, 2011 @11:18PM (#2543407)
The risk you mention can be worked around with loss limiting orders placed on the other side.
Re:The stock market is controlled by algorithms that are fig... by vecter on Thursday May 12, 2011 @11:45PM (#2543469)
You make it sound so trivial and easy. Do you actually believe it's that simple?
Re:The stock market is controlled by algorithms that are fig... by cma on Thursday May 12, 2011 @11:59PM (#2543496)
My day job is making low-latency trading systems. =)
Re:The stock market is controlled by algorithms that are fig... by vecter on Friday May 13, 2011 @08:36AM (#2544388)
So is mine. I develop HFT trading algorithms using statistical analysis. Placing limit orders on the other side is as dangerous as anything else. Are you on the strategy side or pure development side? If you are developing strategies, then you know what you said in no way mitigates most of the risk that we talked about.
Re:The stock market is controlled by algorithms that are fig... by cma on Friday May 13, 2011 @09:09AM (#2544462)
Pure development. So yeah, there's some handwaving there.

Let's say Galileo had had to climb a rickety ladder to the top of a tower at significant risk of falling and breaking his legs. Would that in any way have altered the social utility of his end product?

re: "every serious player in the stock market also has a telescope," see my comments on the Chicago fiber installation elsewhere in this thread.

Re:The stock market is controlled by algorithms that are fig... by mmodahl on Thursday May 12, 2011 @10:46PM (#2543341)
So you'd rather he not invent the telescope? Because inventing the telescope "provides no net-wealth to society?"
Re:The stock market is controlled by algorithms that are fig... by cma on Thursday May 12, 2011 @11:01PM (#2543374)
Lets say it is a hypothetical telescope that has no other use than banana/orange arbitrage. If you don't think that addition makes the analogy more accurate, you haven't seen market-data parsing FFPGAs.

And don't forget this:

"By March 2009 Spread was moving dirt. Soon it had 125 construction crews working at once. [...] At 825 miles and 13.3 milliseconds, Spread's circuit shaves 100 miles and 3 milliseconds off of the previous route of lowest latency. [...] Spread won't disclose cost, but Jason Cohen, the chief operating officer of Allied Fiber, which is building a nationwide network, says laying cable through easy terrain runs $200,000 per mile. Half of Spread's route, however, is through tough virgin terrain, pushing forbes' estimate of its cost toward $300 million."

http://www.forbes.com/forbes/2010/0927/outfront-netscape-jim...

And, back away from the hypothetical, obviously from my post I think Galileo's telescope was great overall.

Re:The stock market is controlled by algorithms that are fig... by mmodahl on Thursday May 12, 2011 @11:19PM (#2543408)
"One day a man invents a telescope .... He provides no net-wealth to society as a whole"

Financial incentive drives creation. Programable gate arrays that can keep a limit order book drive sales of the cards. If more cards are sold, they get cheaper. If they are cheaper, they are more accessible for medical imaging.

What impact on the national fiber network has the Spread line had? What has it taught us?

Re:The stock market is controlled by algorithms that are fig... by cma on Thursday May 12, 2011 @11:36PM (#2543449)
Sometimes the Randian suggestion doesn't meaningfully differ from the caricature of Keynes: "dump a portion of production into the ocean to help the economy."

World War II spurred development of new metal alloys, better vehicles, and thousands of other innovations. Does that make Hitler a hero? Was war the only path?

(Sorry, I had to Godwin it before this thread got too long; the more general war analogy isn't too far off though: $300-million in heavy-equipment laden fiber laying probably resulted in some injury--is it too much to ask that we both get the beneficial externalities of R&D and put in the fiber for a useful purpose in the first place?)

Re:The stock market is controlled by algorithms that are fig... by rwmj on Friday May 13, 2011 @07:18AM (#2544247)
The company I work for (Red Hat) has made a ton of contributions to the Linux kernel to improve real time performance. The driver and money for these contributions is high-frequency traders, but the benefits accrue to all sorts of areas and kernel users.
Re:The stock market is controlled by algorithms that are fig... by nradov on Thursday May 12, 2011 @08:35PM (#2542968)
There's no evidence that the majority of HFTs consistently make a profit. It may just look that way due to survivorship bias. The HFTs that do make a profit are harvesting their gains from other active traders, not from long-term investors.
Re:The stock market is controlled by algorithms that are fig... by yummyfajitas on Friday May 13, 2011 @08:17AM (#2544352)
HFTs tend to earn the spread - this is what it means to sell liquidity. If the bid/ask is $10.00/$10.05, the HFT might make $0.05 on that trade (assuming the market doesn't move before he can execute the other side).

If you don't wish to pay the spread (the HFT's "rent"), use post only orders. Your are guaranteed not to pay the HFT rent, but you run the risk that your order may never be filled.

http://www.nasdaqomxtrader.com/content/ProductsServices/Trad...

Re:The stock market is controlled by algorithms that are fig... by waqf on Thursday May 12, 2011 @05:27PM (#2542391)
This reminds me of my idea for a compiler that guarantees performance but not correctness, instead of the other way round.
Re:The stock market is controlled by algorithms that are fig... by repsilat on Thursday May 12, 2011 @10:59PM (#2543368)
3D rendering and media compression are almost entirely based on the idea that "close enough" is vastly cheaper than "do it right". At the code level, though... You'd have to be fairly careful about what you allowed, and you'd probably have to be able to enforce correctness to some degree to get any meaningful use out of it.

I like the idea of a compiler making lossy simplifications to mathematical formulas for speed - "You told me to divide by seven. This (faster) machine code divides by 8 most of the time, and divides by 4 every now and then." Blindly trusting the branch-predictor could be fun, too, as long as it took into account the correct answer in its subsequent predictions.

Were you thinking along similar lines, or did you have something completely different in mind?

Re:The stock market is controlled by algorithms that are fig... by waqf on Friday May 13, 2011 @05:17PM (#2546225)
I like those ideas a lot. I hadn't thought about it in enough detail to come up with similar ones, but that's exactly what I was hoping for.

The problem, of course, is that it's hard to find a metric for "close enough" that works for every program. Perhaps it's easier if you're compiling from a high-level language than from C.

Re:The stock market is controlled by algorithms that are fig... by regehr on Thursday May 12, 2011 @05:42PM (#2542449)
I have a compiler like that. It turns any code into "return 0".
Re:The stock market is controlled by algorithms that are fig... by btilly on Thursday May 12, 2011 @07:59PM (#2542874)
Re:The stock market is controlled by algorithms that are fig... by fleitz on Thursday May 12, 2011 @07:16PM (#2542738)
Well it largely depends on whether you want price stability or price discovery from your markets.

Please explain where the source of capital for the arbitrage comes from, it's not exactly clear to me who would be willing to take huge arbitrage losses from this market. (Maybe the fed could back it, they seem to like taking a bath on arbitrage)

Re:The stock market is controlled by algorithms that are fig... by r00fus on Thursday May 12, 2011 @05:27PM (#2542392)
This would be a great idea, but will never get implemented because wall street has almost perfect regulatory capture.
Re:The stock market is controlled by algorithms that are fig... by baconface on Thursday May 12, 2011 @09:59PM (#2543213)
Bitcoin is a market that grows at a fixed rate!

The stock market is kind of a joke...