I developed HFT algorithms in a previous life at a very large, well known bank. The majority of trades out there are in fact market making related. The ones that are market taking are usually at the expense of OTHER HFT algorithms, the ones that are slow and showing out dated prices.

And yes, HFT algorithms most DEFINITELY hold inventory. Some don't, but most do. It's too expensive to get hit on a bid and then try to get rid of it right away. They often hold inventory for days, weeks, even months. I know this for a fact. I can't tell you know much time is spent on worrying about and managing the inventory HFT systems accumulate.

HFT is immensely useful for large mutual funds. They replaced a broken system of high school dropouts who worked on the floor of the NYSE. Mutual funds HATED calling the floor, waiting an hour for execution, not knowing what was happening, and inevitably being taken for a ride. This is all public...go ahead and read a lot of the articles after Thain took over as president of the NYSE and gave tours of the floor to the large mutual fund managers...they were not impressed, to put it mildly.

With HFT, you have transparency and immediacy. You put in an order, and you get that price. In the old days, mutual funds called in an order, and had no idea what they would get, it would be whatever fill the NYSE trader gave them. Executing orders throughout the day makes perfect sense, and is cheaper execution in the long term. You simply get a bad price if you try to sell a billion shares of MSFT in an instant, so instead you spread it over time.

HFT does not front run in any way that is different than trading since the beginning of time. They are trying to understand if their is future demand, and adjusting their prices accordingly. You don't think that after being hit on a bid, the old NYSE floor traders didn't change their prices? They're doing the same exact thing as HFT does.

There's a ton more to say about it, but these are just to counter some of your points.

I wonder how much of the dislike for HFT comes from applying intuition about 2 party trades to a many party market. Mostly, I think people ignore that the HFT systems are competing with each other, not just arbitrarily stepping into the middle of transactions. The latter really violates intuition about fairness.

One of the greatest examples of how nothing nefarious is going on is from a large multi billion dollar hedge fund I used to work at. They have both an equity investment group, as well as a HFT equity liquidity providing platform. I worked on HFT, and knew the equity guys. They always wanted to execute with us in the open market. It was so much better than what they used to have to do.

Why would one of the most sophisticated equity investors in the world want to run an operation that "skims" from the other? Because that's not what HFT does, and of course they know that. They use their own product (their HFT system) to execute their own buy orders. If that's not a testimonial, I don't know what is.

This is precisely the point about HFT. If you are a seller, there isn't always a buyer. HFT is there when there ISN'T the other side. The scenario you described is absurd. How often do a buyer / seller want to interact within the same millisecond?

They can of course. HFT does nothing to stop that. Two people can trade in the open market with each other whenever they want. The reason most of the time HFT is involved is HFT gives the best price. That's why it's become so big. They provide the best price to their customer, not attacking some "flaw" in the exchange.

Here's an exciting company with a spread of $0.01 (at the moment anyway):

http://finance.yahoo.com/q/ecn?s=WEC+Order+Book

My selection criteria was to try to find a S&P 500 component that I wouldn't expect to be among the most highly traded S&P 500 components.

Isn't that exactly the point the GP was making?

If it's bad that one firm capitalizing on the situation of another firm making a mistake is the point, then I think he should really rethink his participation in capitalism.

The problem with the competition between HFTs is that it's turned into a ridiculous arms race to shave microseconds off the response time - how is that in any way productive?

It turns out to price an asset, like a stock, a wide variety of skills is needed. Some people focus on macro trends, like analyzing political events, oil prices, etc. Some people talk to management of the company, count cars at Wal Mart, etc. These are the micro trends.

And some people try to estimate the current supply and demand for a certain stock. That is what HFT does. So yes, it has "regards to the stocks they are trading." Just different regards. That's why specialization works so well. HFT worries about current supply/demand issues, and tries to estimate a fair price to set the market so that when the average investor comes in, and looks at a stock with a 1 cent spread, he is going to get a fair price. Over the long term, people who analyze micro and macro factors will trade with HFT, which will then force the algorithms to change the stock price to reflect their views.

Is an arms race to shave milliseconds off of trading times the most important thing in the world? Absolutely not. But is gaming the Apple app store to get your mobile app higher in the rankings any different? At least with HFT, milliseconds do matter in more situations than you can imagine. If one day you buy a stock that had bad news announced just milliseconds before, and HFT did not update that quickly so you paid a price way, way too high, you would be upset. Again, not the most important thing in the world but it does help.

The millisecond thing is highly overrated. Anyone in the industry knows that it only matters to a small degree…understanding supply demand and adjusting accordingly is the most important skill. That's why the most profitable high frequency firms don't care about what millisecond they execute in…that is one of the great ironies of this--you guys are concerned about something that not many other people who actually do this for a living are concerned about. But you guys latch on this minor point as the major one, and that is a mistake. Do developers worry about how quickly their apps load? Yes. Does it mean their product will be a success? Not necessarily. That is the same thing here. HFT shops work to make sure they aren't too behind, but focusing on that aspect is just trivial compared to the actual work they do.

It has always been in the interest of high volume traders to hide their intentions just as it has been in the interest of all other market participants (including other high volume traders) to attempt to discover those intentions. In this sense, trading now is exactly as it has always been, although it occurs at a more rapid pace.

"Otherwise the HFT spots it (and they usually do) and then front runs all of the trades, just skimming pennies off. What use are they?"

Please explain why it is bad/wrong/unlawful/unethical that other traders may act when a large trader does a bad job of hiding his intentions.

"The majority of the trades are not making markets at all, they are just zipping back and forth to collect pennies."

I don't understand. What do you mean by this?

"If they were market makers then they would hold inventory, but they never do."

I assume this to mean that you believe that the definition of "market maker" should include a reference to the length of time one holds a position. If you demand that to be called a "market maker," one has to hold a position for a minimum amount of time, then you will exclude some market participants from being so designated. I don't understand how the definition of this term is relevant to the discussion; please explain.

"NOBODY believes the "liquidity" story."

I assume that by ""liquidity" story" you mean the idea that HFT firms increase liquidity (more size with which to trade, at a better price) and that this is to be valued. Is this not the case?

"and what about flooding the market with fake bids to cause opponents to get overloaded ?"

Deliberately entering unbalanced (one-sided) orders without the intention of trading so as to manipulate the market is illegal. This vague idea is a serious infraction of SEC rules that carries significant fines and penalties. See: http://www.finra.org/Newsroom/NewsReleases/2010/P121951

"or running tracer LFOs spitting out strange bid patterns just to see if they can detect another hidden program ?"

I'm not familiar with "tracer LFOs," please explain what these are. If they are orders or order submission strategies intended to gauge market interest, do you think that they are bad/wrong/unlawful/unethical? If so, why?

So when BigMutualFund decides they want to buy 50 Million Shares of MegaCorp they try to disguise their order so that the share price of MegaCorp doesn't go up to much as they make their buy. That's absolutely right.

You're also absolutely right that HFTs have made it harder to disguise this action. The HFTs "see" what's going on and start buying shares of MegaCorp at a higher price and then selling them to BigMutualFund. So BigMutualFund makes a bit less money.

So where does BigMutualFund's go? To the HFTs? NO! The HFTs aren't changing the spread width, so they aren't making more money.

You know who makes more money? The current shareholders of MegaCorp who decide that they want to sell! And that's great! Whatever information it was that made BigMutualFund think that now was the time to buy gets communicated to the market faster (in terms of the share price) because of the great work the HFTs are doing.

More efficiency for all!

They can do the trade slightly faster, and according to the rules of the market, if you place an order to sell with a limit of $50 then someone willing to buy now for $50 beats someone willing to buy a second later for $51. If that's not what you wanted, you should have made the limit $51.

Maybe we need new order types, but that would set off a slew of complaints about additional complexity. And note that eliminating the sub-penny rule would put a lot of the front-runners out of business, since they'd only be making $.001 per share rather than $.01.

We have misinvested countless millions on a system which produces the wrong answer, optimizing for millisecond latencies which benefit neither companies needing capital not investors providing it.