Furthermore to me HFT looks like stealing because it is very inefficient relative to the need. If exchanges were to reorganize how they work slightly, we would get much better price stability, somewhat less liquidity, and have a much lower premium.
The necessary change is simply this. Exchanges would have to set a price at which orders are to execute. Whenever there are excess orders in one direction or the other, that price would drift up or down, slowly, until the order executes. The result is that trades might take a minute to execute, but margins would be far less. Sure, a ton of finance people would jump up and down screaming bloody murder about how many of their complex trades would be affected. But my opinion is that that economic activity takes a lot of brains and resources, and doesn't significantly add to the purported purpose of finance - to make money available to companies engaged in useful production.
How would this even work? Understanding Market micro structure is literally my day job and I'm having trouble parsing your proposal.
Consider the market where I want to buy at a limit of 9.90 and someone else wants to sell down to a limit of 10.00.
What price would the exchange set? People have already said the maximum the market will buy for is 9.9 and the minimum the market will sell for is 10.
if there is no overlap you just cant' trade no matter what the exchange does.
Today the price is the price at which the last transaction happened.
I am proposing instead that the price is set and updated by the exchange, and transactions only happen when they can execute at the exchange price. And the exchange price moves at a preset rate - down if there are outstanding sell orders, up if there are outstanding buy orders, else still.
So in your situation the price would wind up somewhere between 9.9 and 10, then stay indefinitely until another order began moving the price.
Basically for someone placing an order you're losing the ability to trade instantly, and gaining the prospect of trading at a possibly better price. Therefore you don't need a pool of people trying to give you instant trades.
How does HFT change that?
Now the spread is 8 cents. Anyone who wanted to buy can now buy it at a penny cheaper than an instant ago and the opposite holds true for anyone wanting to sell.
Isn't this only true if you measure price stability based on amount of changes in the price, but not magnitude of those changes? How can they lower the spread but simultaneously decrease stability?
See the flash crash for the best known example.
The old time finance guys have no idea what a back-propagating multilayer neural network is and how it's eating their lunch, and there is resentment and suspicions of foul play. I am more suspicious about the suspicion. You can pitch a tent and set up a hedge fund like Renassiance or DE Shaw without the blessing of an investment bank, and perhaps the banks resent this, despite the fact that they're doing HFT as well.
As for the argument that the value they provide is "liquidity", that also seems dubious. I've seen that argument thrown around a few times now, but I'm not entirely convinced that HFTs, as they exist and work now, are achieving this goal fairly, nor that this is necessarily the best solution.
(I'm told by knowledgeable people that there are indeed some shady things that happen in HFT, such as quote stuffing, that ought to be fixed, which makes sense. But I don't think that brings us anywhere near "shadow government", nor does it seem like a damning problem for HFT as a concept.)
Unless you can say that the "price extracted" is fair and self regulating without any clear and transparent oversight, I think there is probably room for improvement and a benefit from increased scrutiny. However, as I said, my knowledge of the internal workings of the markets is still pretty limited.
Paper on frequent batch auctions: http://faculty.chicagobooth.edu/eric.budish/research/HFT-Fre...
If it is FIFO then you still have a latency race.
"There are some subtleties involved [...] since it takes light roughly 4 milliseconds to travel between Chicago (where ES trades) and New York (where SPY trades), [...] We would also like to suggest that the fact that special relativity plays a role in these calculations is support for frequent batch auctions." (emph added)
In particular: you can come away from Lewis' book believing that the markets prior to electronic trading were reasonable fair, when the reality is the opposite: the trendline of fairness and transparency in the market is sharply positive, and the inflection point of that trendline is the advent of fast electronic trading.
Remove the liquidity (which comes at quite a steep price it seems), now what? There seems to be plenty of capital looking for investments.
The immense effort spent on HFT could now be concentrated on a) finding things worth investing and finding better ways to do so b) create more things worth investing in
It's a little bit like saying the guy who runs your 7-11 is a scumbag because he charges an extra 50c for a carton of milk, and "who needs that liquidity, they could just wait until <insert large supermarket chain> opens and get their milk there". Sure, but they might want the milk now. And no one forces them to get it from the 7-11. It's a free market.
But more importantly, HFT is not the same as liquidity. HFT only works in the first place because there are already market orders being placed; if no one but HFTs wanted to buy or sell stocks, no trades would take place. So the HFTs, by themselves, can't provide liquidity; they have to wait for someone else to do it, and then execute their trades to help narrow the bid-ask spreads.
I'll agree that there is some value to narrowing the bid-ask spreads; but is it enough value to justify the enormous effort and the enormous amount of human talent that goes into HFT, that could be doing lots of other valuable things? I don't think so; I think all that is a sign that the incentives in our society are screwed up.
HFTs by themselves can provide liquidity (like any other investor), namely by entering a limit order in the order book. They can also by themselves narrow the bid-ask spread, namely by entering a limit order at a price more competitive than the current first limit.
Of course, someone else has to send a market order to the exchange for a trade to execute.
Yes.
> HFTs by themselves can provide liquidity (like any other investor)
Exactly: like any other investor. Meaning, the "high frequency" part of HFT is not what provides the liquidity. (I admit that I should have made that clearer in my previous post.)
Suppose an HFT enters a limit order that is more competitive than the current limit. Either some other investor would have been willing to trade at the same price (as the HFT's limit order), or not. If not, then it doesn't matter how fast the HFT places the limit order; it's going to increase liquidity in any case. So the "high frequency" part is not necessary; the key is that the HFT is an investor willing to trade at a certain price.
But if some other investor would have been willing to trade at the same price, then the only difference the HFT makes is speed: the liquidity gets added sooner than it otherwise would have. So the "high frequency" part doesn't change anything except who the profits from the trade go to.
However, it's the category of sell-side traders in general that matters for liquidity.
You're using imprecise language here (for instance, referring to market makers as "investors"), but the clear implication is that you believe that normal, "buy-side" investors provide adequate liquidity. But they don't, and we know that, because the more competitive the sell-side gets, the lower spreads get.
Well, of course that depends on how you define "adequate". What is all this incessant trading of stocks for? What value does it provide? For example, if I'm an investor (a "real" one, not a market maker, to use less imprecise language) with a long time horizon, something like 30 years, because I'm saving for retirement, how does HFT make me better off?
In other words, brokerage fees, or the equivalent, should be lower, so the overhead to execute a trade is lower. That has some value, yes, but not a lot, because a buy-and-hold investor, of course, doesn't execute many trades, so the overhead cost of executing trades is already pretty small for him.
Also, most "buy-and-hold" investors hold mutual funds, not stocks, which changes things. See below.
> costs...can be very important even to value investors when there is some extrinsic prompt to trading
Yes, but as I just noted, most "buy-and-hold" investors are holding mutual funds, not stocks, so they aren't paying the direct costs of stock trading anyway. When I want to rebalance my 401k, I don't trade stocks, I trade mutual fund shares, and they're all shares of different funds offered by Fidelity or Vanguard or whoever my 401k provider is. (And with most 401k's, certainly with the ones I have, as long as you don't rebalance too often there is no fee for rebalancing.)
So decreasing the overhead cost of trading will only appear to me, if at all, as a decrease in mutual fund fees; but with most 401k's the individual doesn't see those anyway, because they're provided through employers. I don't know how much the fee question affects the negotiations between employers and mutual fund providers for setting up 401k's, but in any case that's at least two layers of indirection between me as a retirement investor and the overhead cost of executing individual stock trades.
So I can see some small benefit, yes, but is it enough to offset the high social cost of having so many smart people doing HFT instead of something more productive?
Whatever you pay to your brokerage, you pay on top of the spread. The spread is what you pay to place a market order.
In exchange for the privilege of buying right now, you pay the best offer price. In exchange for the privilege of selling right now, you pay the best bid price.
It's a commission you pay per share; the more shares you try to move, the more you pay whoever's providing you the liquidity.
I agree with you for the case of a "buy and hold" investor buying or selling actual shares of stock (with the proviso that the overhead to such an investor is already pretty low since he doesn't execute many trades, so the gain to him from something like HFT decreasing the bid-ask spread is small). But as I noted, many, probably most, "buy and hold" investors hold mutual funds through 401k's, not individual stocks.
For example, I have a 401k. I don't place market orders. I rebalance my portfolio every so often, which, after a lot of intermediate steps, might result in some mutual fund manager placing a market order. But I don't see any of the direct costs associated with that, whether it's paying the spread or anything else. I only see the net overhead of the mutual fund as a whole. (And, as I noted, for a lot of 401k's, like mine, I don't even see that, because the 401k mutual funds are no-load.)
Whether it is a direct line item on your prospectus or not is immaterial. No load mutual funds are more successful when they have lower execution costs. The biggest driver to low execution costs is an electronic market and then the bid/ask spread. HFT enable both of those low costs.
I see how HFT can narrow the bid-ask spread, but how does HFT enable an electronic market? Isn't it the other way around?
Imagine that there is a "true price" of some security at price x, and your bid and your offer are positioned around that true price at x - d (your bid) and x + d (your offer). So your spread is 2d. Then some external (public) event happens, which moves the true price by an amount greater than d. Let's say it is a big event, and the true price falls by 4d. Now there is a race. If you don't manage to cancel your bid in time, a predator will 'pick you off', take out your bid, and they'll sell to you at (x - d) when the true price is now (x - 4d). The predator has made an immediate (paper) profit of 3d, and you've made a loss of the same amount. If you're fast enough, you can successfully cancel your bid and repost it further down the book at x - 5d.
Now, as a market maker, a liquidity provider, you could increase your safety margin by quoting a wider spread, setting your bid and offer at +- 2d instead of +- 1d. Now it takes a larger move before you're in danger of being "picked off", but you'll also attract less customers and make less profits. Speed (latency) is directly correlated to how tight a spread a market maker can quote.
If you assume that the number of 'real' trader (not middlemen) who want to buy or sell is constant (not necessarily true, but assume it for a moment), then it stands to reason that the tighter the spread, the less the market makers are profiting. At a small enough spread, they make no profit, since the profit from the flow on both sides is cancelled out by the losses from adverse selection (trading when they couldn't cancel in time - being victim to a predator). But if they can increase their speed, and minimize their adverse selection losses, then they can quote tighter and still make a profit. And do it with machines, and cut overheads of hiring humans, and you can quote even tighter since you need less trading profit to make a net profit. So this is the value generated, automation + speed = the smallest amount of frictional costs being extracted from the market.
Ok, this makes sense. But it still leaves the question, how much are the frictional costs decreased by HFT, and is that benefit enough to offset the social costs?
Such as? All I heard was smart boys and girls go and do a quantitative finance degree instead of Solve The World's Problems degree.
And that's bullshit. The social cost of _not_ providing other, better alternatives to our youth (downsizing NASA, underfunding research and scientific endeavors), lack of patent reform, lack of bandwidth and unified cross-country 3G/4G has much worse costs.
No, they go and do a Solve The World's Problems degree and then end up in quantitative finance because they can't get rich solving the world's problems.
> The social cost of _not_ providing other, better alternatives to our youth (downsizing NASA, underfunding research and scientific endeavors), lack of patent reform, lack of bandwidth and unified cross-country 3G/4G has much worse costs
I think we're in violent agreement. This is exactly the social cost I was talking about. Our society is run by people who can't be bothered to compensate someone properly for curing cancer, but who will throw millions of dollars a year at someone for shaving a few microseconds off an HFT algorithm. That makes no sense to me.
Why are things this way? Lots of reasons, of course, but one of them is that, whenever anyone suggests that maybe it would be better if smart people were directed into more productive endeavors, the financial people have kittens and say it's going to wreck the world's economy if we don't shave those few microseconds off the HFT algorithm, because there won't be enough liquidity or something like that.
And that is bullshit. Sure, if we were at a point where all the big problems were solved, then yes, making tiny incremental improvements in the price of liquidity might be near the top of the priority list. But as things are, it should be somewhere around 56,732nd place. The problem to be solved is that there is huge, huge value to be captured by things like curing cancer, much, much more value than there is to be captured by shaving microseconds off HFT algorithms, but nobody knows how to capture it. How is making stock trading a fraction of a percent more efficient going to fix that?
Is that enough of an advantage to outweigh the cost of the small number of intelligent people in HFT? I don't know, but it certainly seems easier to justify than the phenomenal amount of capital spent on internet ads. I'm a bit of a free market capitalist, so my biased response is how else should we allocate people's output?
I don't actually have a sense of how big or small the number is as a percentage of intelligent people. Particularly when you observe that, as you say, HFT is just a tiny part of the finance industry. There are many other parts of that industry that are also sucking up smart people to do things that, to me, have very little value compared to the other things those people could be doing.
> how else should we allocate people's output
Sorry to redirect again, but see my response to pas upthread. :-)
So it's hard to have the "it's terrible so many smart people are doing X" argument when our society has made it pretty obvious we value selling internet ads as high as any other possible commercial enterprise.
Me, I'll take the other position, which is that yes, the markets are bad at allocating capital, but they are better than our alternatives, and the silly little pocket change inefficiencies that go into speculative market making are a small price to pay for all the advantages of a modern liquid, price efficient market.
How are you getting that from what I said? You could s/HFT/internet ads/ in my post in response to pas and it would be pretty much the post I would have written about internet ads if that had been the subject under discussion. (Well, not really, because you'd have to change other things as well, but hopefully you see what I mean.)
> markets are bad at allocating capital, but they are better than our alternatives
This I agree with. But markets are as bad (or good) at allocating capital as the people who participate in them. In other words, markets reflect the values of the people who trade. I wasn't ranting about markets not being able to properly compensate curing cancer because markets are bad; I was ranting about people's values being so screwed up that the price signals they are sending into markets are making people rich for HFT and internet ads but not for curing cancer.
It's implied by what you didn't say. You don't require technologists involved in internet ads to justify their existence whenever a post about google comes up. Nor when news breaks of a new internet startup being acquired. I'm picking on you a little bit as a proxy for the entire HN community, as there seems to be a bias against HFT and towards internet ads that is a little unseemly.
Further, I'm not convinced you understand the scope of money involved in HFT vs cancer treatment for instance. When Roche bought genentech (a cancer research company) they paid 46 billion dollars for the privilege. Knight Capital Group, one of the biggest players in HFT is worth around 1.4 billion dollars. The market seems to be allocating resources correctly. The PR companies on the other hand...
Wow, you've gone through my entire HN corpus to verify that?
> I'm picking on you a little bit as a proxy for the entire HN community
I would prefer that you didn't do that, since I am not a good proxy for any community that thinks figuring out ways to get people to click on ads is a good use of smart people's time and energy. (Btw, I'm not sure that the entire HN community is such a community; a part of it may be.)
> The market seems to be allocating resources correctly
The numbers you give are interesting, but I'm not so much interested in money as in human resources. Genentech may have cost $46 billion, but are they paying smart people enough to get them to do cancer research instead of becoming quants?
Some of this may be a perception problem, as you imply by your comment on PR companies. What data I've seen on where smart people can get paid the most indicates that it's not cancer research. But I'm sure the data I've seen is incomplete; I would love to see more comprehensive data if there is any.
This (false) assumption is propagated regularly by articles and books wherein the same pattern repeats itself:
1. limit orders are defined
2. "... but on that morning Joe Trader was shocked to see that his limit orders weren't working!!"
3. description of Joe Trader NOT using limit orders ... or using them and canceling them and chasing prices ... or whatever.
I've seen this pattern in a lot of media since the Lewis book. In reality, limit orders haven't changed and you can set a limit and go to sleep / go on vacation, just like you always could.
Because NOW is when we have the most actionable knowledge.
If you don't buy now, what if the price never goes back down? You missed a buying opportunity.
The 7-11 provides the trade functions "assortment of goods" and "transport" and as such is valuable. The analogy to HFT does not hold.
"Large supermarket chain" also provides the trade functions "assortment of goods" and "transport", so what exactly are you paying your extra 50c for? Liquidity.
What a bunch of assholes those 7-11 types are. Charging us an extra 50c for milk. The nerve.
However, I'm afraid your proof-by-analogy does not work. If you actually know what you are talking about, why not explain the current financial system clearer and with more information?
In turn I promise not to burden this discussion with a dissection of your analogy, or even my own set of analogies which would beautifully fit my current bias, and which you would surely reject outright.
Market makers generally, smooth these time disparities. As long as the smoothing is cheaper than the price disparity then it is a net positive for the "real" participants. How do we determine if this is cheaper? Quite simply, the market! You can trade non-liquid products electronically. The price spread his much higher in those products.
The price of liquidity is less than the inefficiency (loss) that comes from an illiquid market (people who mutually want to execute a trade but cannot). This is rather straightforward to demonstrate mathematically.
Just because HFT extracts a premium (similar to a transaction fee) on the spread doesn't mean both parties (and the rest of the market) can't still benefit.
> The immense effort spent on HFT could now be concentrated on a) finding things worth investing and finding better ways to do so b) create more things worth investing in
This is analogous to the argument often levied by non-techies at the entire tech sector: "They could be developing software to cure cancer, so why are they creating more apps to send selfies to their friends?"
And IMO that's often a good argument, though it's not really an argument against the techies but against our society as a whole. The fact that techies can get rich making selfie apps much more easily than by curing cancer, is a sign that incentives in our society are screwed up, in much the same way as the fact that smart people can get rich by running HFT firms much more easily than by running companies that directly produce goods and services.
The "premium" could then be split between the both parties. The cake is larger (but maybe not split as evenly).
The argument about webapps vs. curing cancer is also levied by techies right here on this site and might be valid, (yet weaker, since webapp-big data might eventually help with medical-big data?). Either way you have not made an argument for HFT with your last sentence.
For those like me who don't understand, what does it mean? What are the benefits? What are the drawbacks?
A very illiquid market, I want to buy for $9, somebody else wants to sell for $10, nothing else. And now? How does HFT help? How does a traditional market maker help? He buys for $10 and sells for $9 and then somehow recovers the loss?
(Further down tptacek posted a great analogy. https://news.ycombinator.com/item?id=7853500)
People are more likely to trade if they feel a lower risk of being stuck in a 'bad' position.
Many times, exchanges actually pay market makers to provide liquidity to try and kickstart a particular market.
So you and the seller see better offers than if they didn't exist, even if those offers aren't good enough to get you to trade.
The more liquid something is, the closer it is to cash.
Think about your car -- if there was a public price for your car and you could decide at any moment whether you wanted to sell for that price, or close to that price, that would be a benefit to you. But instead selling your car is quite a chore and you are never sure whether you are getting ripped off or not. A liquid market has a public price and a degree of fairness and confidence baked into that price.
In order to provide liquidity market makers agree to buy stocks from anyone who will sell them, and then take care of finding the buyer on the other end. The difference between what they buy it for and what they sell it for is called the 'spread' and it is how market makers traditionally have made money. The spread is also the fair fee they charge for the service they provide, which is liquidity, and the risk they take. If market makers did not know for sure that someone else would be willing to buy the stock right away, that they might have to sit on it, they would be taking on a very large risk. The more people that buy and sell, the less real that risk becomes.
HFT makes their money, generally, by front running. Front running is the practice of buying a stock because you know the California Pension Fund is about to buy it, and so you know the price is about to rise. You can immediately turn around and sell it when it goes up.
Front running by a bank is illegal, but it is widely suspected that many banks engaged in the practice for years. It is illegal because the bank is supposed to be acting on their client's behalf and in their interest, so when they are trusted to place the order, they first buy some of the stock for themselves or for the bank's inside funds. It is basically insider trading.
When HFT engages in front running, they have detected that an order is coming down the pike, and they jump in front at the last nanosecond. The reason they spend so much money trying to do this is because it is essentially fool proof when done correctly. You can argue that by not taking any market risk, and not holding the stocks, they aren't providing liquidity, they are parasites on real stock traders and they are essentially using inside information. But its not that cut and dried because there is almost no way to eliminate the problem. At least HFT reduce the time window during which someone's signal to purchase something can be exploited.
It used to be that humans managed all of this. Large brokerage houses would be responsible for executing these trades, and it was common practice for brokers to tip each other off when they received large orders. These tips were very valuable because they guaranteed a winning trade. If you ever wondered how meat-head frat boys could make millions of dollars on Wall Street despite not being especially intelligent, this was a big part of it. At least now the people who are scalping everyone's orders have to be brilliant.
This is not possible.
For example, you want to buy 10,000 shares of X.
Matching engine finds 100 at 9.98 - it’s the NBBO (best buy offer), it’s required to execute that, and carry on looking for ways to fulfill your order below or at your bid.
Meanwhile, the HFT offering those 100 shares sees this, decides its a parcel of a large buy, and if it has a quicker route / algo than the matching engine to other exchanges, goes there, executes immediate buys at say, 9.99, then immediately raises their price to make a profit on the spread.
What is actually happening is that the HFT is offering shares on all the exchanges on both sides of the bid/ask spread at the same time. When they get filled on an order at a price on 1 exchange it is used as one of many signals to indicate if their pricing is accurate. If their algorithm thinks that fill indicates a price movement (for instance if the entire level was removed) then they will update their prices on all the other exchanges accordingly. If the algorithm doesn't believe this indicates a long term price movement they won't (or they may even re-up at the same price on the exchange they got filled on).
No one is going out and buying ahead of someone else because that is too expensive (you have to pay the bid/ask spread to do it) and too risky (you might be wrong about your price determination).
Finally, just a detail, you can typically opt out of NBBO matching if you want.
Basically, with more participants, the bid-offer spread will begin to close, and trading will occur at more prices.
At any given point in time, the "true" (buy-side) investors are likely to be relatively far apart on prices. This is especially true in thinly-traded markets. For example: consider housing, which is the canonical example of an illiquid market.
I know the value of my house (it's around $300k). Say I want to sell it. The more liquidity I need, the worse price I'm going to get. Anyone with any intuition for the housing market knows that if I have to sell my house tomorrow, I am going to get a god-awful price for it; the "tomorrow" price for my house is many tens of thousands of dollars off its true value.
This creates huge problems for homeowners. The obvious standard advice for sellers is to wait patiently for the best price, counting on many weeks or months before a reasonable offer arrives. But I have to pay money every month I hold on to the house. If I need to move immediately, for instance for a job, or because my financial circumstances have changed, I might need the house to sell quickly, and because houses are illiquid I have to accept a crappy price. Or consider trends in the market: by forcing me to hold the house for months rather than days, I'm maximally exposed to swings in the market. So if Chicago housing prices crater while I'm trying to sell, the extra time it takes to unload the house takes that price swing out of my hide.
(You can easily see how the same thing happens in reverse in "hot" markets like Palo Alto, with the buyer now assuming the role of the hapless Chicago seller).
The exact same thing happens in the public markets; it's just not as intuitively obvious because we're working in smaller deltas of time and price.
But the public markets have a huge advantage that the real estate market doesn't: market makers.
Imagine if houses traded (were bought and sold) so frequently that a smart company could make good educated guesses about the current value of any given house. Imagine if that company would on any given day snap up a house offered for sale. That company wouldn't pay my asking price for my Chicago house, but it would pay something much closer to it than the "true" market would. After buying my house, the company would immediately offer my house for sale at any price greater than what it bought the house for. I wouldn't care, though: either I'd be thrilled for the opportunity to sell my house quickly and painlessly, or I just wouldn't accept their offer and instead do what I do now, which is to wait for the best offer.
Assuming the housing market makers were smart, the money they'd get by forcing me to accept a lower bid would be "free money" they get simply by being smart about valuing houses and having capital available to deploy. We'd all be a little irritated at them for scalping distressed buyers and sellers. But something else will inevitably happen: other smart firms will smell the free money, and they will compete for it. To capture a share of the premiums, all they have to do is offer a slightly more favorable price, so that's what they'll do. Over time, the money the housing market makers will get less and less "free", and the price penalty for immediate liquidity will get lower and lower.
In my fantasy real estate market, I now have the opportunity to avail myself of a reasonable "market" order, or to buy or sell on a "limit" price. I could do that if there were housing market makers, but in their absence I can't, because the price hit for selling tomorrow is too great; it might be a 25% discount on my asking price, or even something close to 50%. That's the cost of illiquidity, or, stated more directly, the value of liquidity.
It's probably also worth saying that liquidity is a presumption of the public electronic markets. Because it's taken for granted, there are whole trading strategies (hedging, for instance) that depend on its presence. These trading strategies face execution risk: if they can't bail out of a position within a specific window of time, they incur losses for the trader.
(It is because of comments like these that I have your comments page bookmarked the same way I would a blog.)
Instead of competing to save you a few more slivers of a cent per share, they compete to fill your order a few microseconds faster (at the next-highest price permitted). Seems most traders would prefer the money.
I always see this argument. Market makers provide liquidity, and yes, HFT often act as market makers. However, I am not at all convinced that the high frequency element adds anything useful to the equation. I don't care whether my trade executes in a microsecond or a second. In fact, I often leave limit orders open for DAYS in order to get the price I want. I am also not convinced that most HFT are truly acting as market makers.
I think the most charitable thing that can be said about HFT is that it narrows the spread. In order to make it worth their while, the HF traders then have to make a lot more trades. Whether this is adding any real value to the system is harder to say because while the spreads are smaller, the volatility is higher.
I read a copy of the Mentat prayer that had been written for traders somewhere but have lost track of it, that was something like 'by liquidity I give my transactions life, by speed I give them value, by value I make me wealthy' or something along those lines. It was much more clever than I could come up with, but modest google fu has not revealed its location.
Not quite. HFT uses information that you probably don't have, and acts on that information using software and hardware that you probably don't have[0].
You can obtain that information and that software/hardware - it's just expensive (high barrier to entry)[1].
As explained above, HFT exists because portions of the market are not perfectly competitive in the economic sense[2], but that's always been the case (and arguably was more so before the advent of computerized HFT).
[0] It's not so much that you don't have the information - it's that the latency is too high for most players.
[1] At least some portion of these costs are very real - the cost of maintaining low-latency connections between exchanges, colocating hardware, etc.
What value did they add to the market? Bringing buyers and sellers together? Nope. Provide capital to the market? (Which is why stocks supposedly exist...) Nope. They end every day owning nothing.
Lewis's book also shows that many banks and brokers instead route some trades through their own internal 'markets' first, trying to find a match, instead of just trying to get the fairest price available.
Everyone gets screwed. There was a HFT trader who boasted that they went years without ever having a losing day. The only way to do that is in a market that isn't fair, where not all parties have equal access to information.
http://www.chrisstucchio.com/blog/2012/hft_apology.html
http://www.chrisstucchio.com/blog/2012/hft_apology2.html
http://www.chrisstucchio.com/blog/2012/hft_whats_broken.html
https://news.ycombinator.com/item?id=7546104
Can you respond to my post in that subthread?
(1) Someone wants to move a block of 50 shares, but there are only matching orders in the order book for 25 of those 50 shares. The seller splits his order so that the 25 shares move; then either the other 25 sit there until a matching order appears, or the seller changes his offer price for the other 25 shares so that there's a match.
An HFT in this scenario is no different from any other investor deciding whether or not to match an order in the order book; he just moves faster. And since in this scenario there are no other matching orders in the order book without the HFT, the HFT adds liquidity in this scenario.
(2) Someone wants to move a block of 50 shares. There is no matching order in the order computer in which they place their sell order, but there are matching orders in other computers. The HFT sees those other matching orders before other market players, and matches them. Then he places his own order in the computer where the original 50-share sell order is, and matches it so it executes.
In this scenario, the HFT adds no liquidity; all he does is shift the profit on the trade from other market players to himself, by taking advantage of the latency between different computers on which orders are placed. The trade would have happened anyway once the two computers reconciled their orders.
The reason is precisely because HFT market makers are fast and sophisticated enough that they want to trade with all small block buyers, so they are resting orders at every trade-able price on every exchange they can.
The speed arbitrage comes from the fact that if they see a level being removed at one exchange it is a demand signal, so they change their own prices on the other exchanges. This is faster and cheaper than going out and sweeping another level (which requires paying the spread).
Essentially, there is no liquidity difference in this case, as all the same liquidity providers are there. They are just pricing their liquidity more accurately.
That seems to be a disputed issue right now; claiming that scenario 2 does happen in practice was, as I understand it, one of the main points of Michael Lewis' latest book on HFT. He may be wrong, but I don't think it's as simple as saying "they're so fast that they don't need to".
An HFT may have orders at every tradeable price at one instant, but what about the next instant when the set of tradeable prices changes? On an exchange with multiple computers executing trades, that is precisely the time when scenario 2 can come into play, and the fact that HFTs are faster than everyone else is precisely what gives them the ability to play scenario 2 in this situation.
> if they see a level being removed at one exchange it is a demand signal, so they change their own prices on the other exchanges
To be clear, my scenario 2 was not talking about matching prices on different exchanges. It was talking about a single exchange that has multiple computers executing trades. Conceptually, the single exchange is supposed to have a single order book, but because there is unavoidably latency between the multiple computers, computer A's current view of the order book may not match computer B's. An HFT that can detect the mismatch faster than the exchange's own computers can, can run scenario 2 in that situation.
Let me come right out and say it. Michael Lewis is not "wrong" because he never actually says that scenario 2 happens. The reason he doesn't say that, is that there is no proof that it happens. The reason he couldn't find proof is that it doesn't happen. Any cursory examination of the HFT side of the industry will show that it doesn't happen. He never even does this cursory examination. Go through the book, catalog when he talks to the HFT side. He doesn't do it. The closest he gets is interviewing a developer who is being persecuted by his own buy side firm. Michael Lewis implies that something happens but never actually shows it happening. He does this either due to incompetence, malice, or prejudice.
To prove my point, let's do a thought experiment. Let's say I have 2 HFT systems, each of which are equally fast and are equally good at determining if fills indicate true price movements or blips in the market, and are equally good at evaluating outstanding risk. HFT A's strategy is to quote "small" size in a single market and then when it sees fills it thinks indicate price changes to buy a level in order to raise the price in a particular market. HFT B's strategy is to quote at every level in every market and when it see's fill's that indicate price movements cancel it's order's that are at a bad price.
If their price discovery or risk algorithm are perfect the best case scenario is that HFT A and HFT B are paying the exact same price for the right to trade. If there is any slippage in their price discovery or risk algorithms, HFT B wins as it is not paying for the privilege to trade. QED, algos that don't pay up to jump priority queues do better than algos that do. In the blood thirsty world of HFT market making this inefficiently quickly leads to HFT A going out of business.
As far as your scenario about an exchange that is open to distributed systems attacks, that is akin to asking about a casino that has a secret flaw in their poker shuffling. Yes, that would give an unfair advantage to anyone that found the flaw, but it would also kill their ability to host poker games if it became public knowledge. IE exchanges have a huge incentive to make sure this doesn't happen.
> algos that don't pay up to jump priority queues do better than algos that do
It looks to me like all of this is addressed at the version of scenario 2 that I was not talking about, the one where the HFT is working across multiple exchanges.
> exchanges have a huge incentive to make sure this doesn't happen
I'm not sure the casino analogy quite holds here, but I'll have to defer any further comment until I can find the article I mentioned somewhere in this subthread, that went into how this scenario would work in more detail.
kasey_junk seems well-versed.
HFT is like a litmus test for HN. It is a renewable resource.
1) If you were doing this through a broker, and that was the state of the book, then they would just route your order to BATS (if I'm not mistaken that's a legal requirement for brokers). 2) If you instead placed the order to be on Nasdaq-only, that would lock (have buy and sell orders at the same price) the national book, something which is not allowed due to Reg NMS. So Nasdaq could not display that you want to buy at $10.00 anyway, and (I'm not sure how they implement their stuff) either hide your order or display it at another price.
So I'm not sure how exactly you're framing this situation. If the HFT's computer can see that your order exists, it's because it posted. If it posted, it could not have had the limit price that would've locked the book.
I may be misunderstanding what you mean when you say at the end that the "two computers reconciled their orders" - because nothing like that does happen. If they are computers at the same exchange, then no, that's simply not how matching engines work. If at different exchanges, then there is no reconciling, your order would not have posted at the right price, or your broker would have not tried to get you best execution on your order(which they are bound to doing).
No; as I responded to another post in this subthread, I mean multiple computers belonging to a single exchange, such as NASDAQ. In one of the previous HN threads on this topic, somebody linked to a post that showed the situation I'm talking about; I'll see if I can find it.
HFT seems like stealing only because they are using advanced technology methods to make money, whereas in the 1980s and before market makers routinely manipulated the spread and pocketed likely a similar percent of trading profits.
At least the online brokerages and pioneers of HFT broke down the antiquated 1/8 stock price ticks and lowered the spread and per share transaction costs.