Plus, some of it is unavoidable if you don't have a single unified exchange. Where there's latency, there's inefficiencies, and where there's inefficiency, there's profit to be made. And competition among exchanges is healthy for the ecosystem, so I don't think we'd want to consolidate.
And lastly, HFT has consolidated so much that I don't think it's worth worrying about. Virtu literally switched sides and make most of their money on order execution. Industry-wide, HFT revenues are down like 80% over the last 5-8 years. Between wholesaling/PFOF taking non-toxic order flow off the lit exchanges, and banks finally wising up on not being pants-on-head about their order execution, it's literally just sharks in the pool now, there's not even any water.
Shitty situation either way.
Shares are not held at an exchange.
Besides, when has an exchange gone done for hours, let alone days? Absent intentional breaks in trading as speedbumps.
Why would you assume that.
Every exchange is a valid place to trade and the sip ensures you always have the correct NBBO
Also, "exchange competition" is a thing in US stocks, but not, for example, in the futures' market. And there's HFTs in futures too, so having a single exchange wouldn't "remove" HFTs (not that you'd want to).
That's not true according to their filings. In their filing for 2022 Q4, I see 185m from market making, and 89m from execution.
It could be all run once a day if it was just to give ability to raise funds via investment
But really, the key thing is that liquidity provision actually is a service -- market makers are basically selling insurance. There's money to be made doing his, so people will compete to do so. That's what the millisecond race is about. Faster trading -> less risk for the MM -> less capital needed -> lower profits acceptable. Contra what you're claiming, the race is cutting into profits, not raising them.
If you want to reduce market-maker profits, crack down on payment for order flow, and let everyone compete for a chance to trade against it.
HFT has consistently driven economic costs lower and lower and lower. Of course people can point to bad actors and apples here and there, but overall it performs a valuable economic function which the alternative suggested would not do as efficiently.
To understand this requires understanding markets very deeply and can’t be debated using liberal arts rationalizations.
Can you explain what that is?
> To understand this requires understanding markets very deeply
Apparently you aren't the only person on this forum who does. Are we just supposed to take your word for it?
The only parties with semi-global visibility are prime brokers by definition; they see every position of everyone who custodies with them.
The parties with global visibility of the US market are TRF (trade reporting facility) and those are the only parties who can sort of evaluate the HFT claims without bias or vested interest. Most of the studies in the field have some sort of an angle or vested interest so it’s hard to evaluate the veracity of the claims one way or another.
A counterpoint to HFT is that stock markets existed before the the advent of computers and they had runs, panics and blow ups just like regular markets do now.
In the meantime what I can say very simply in the hope that someone else knowledgeable can contribute earlier.
It’s simply an empirical fact that the costs of intermediation to the system are the lowest they have ever been. The US and other global electronic markets are incredibly efficient and deliver unmatched liquidity, information efficiency, and the lowest costs to the entire capital markets than at anytime in history. That march forward is continuous and brutally competitive
There are many many way to see this and measure it, any serious quantitative analysis, by professionals, for instance trained in econometrics and with access to the raw data, like those at say the Fed, or those operating exchanges, as examples.
As a practitioner who worked on the systems you’re citing I can tell you’re wrong on many fronts.
As far as costs are concerned, you’re not answering the most relevant question that applies to most users of this forum: costs to retail traders, and are only tackling the institutional side of things. Nobody from retail concerns themselves with costs of intermediation because those costs are irrelevant to retail. Not at the order volumes that don’t even incur slippage.
Plus, fees are only part of the game. I can give you an NBBO improvement now to be compliant with the regulation, but I am not guaranteeing anything in terms of price in the next second. See where this is going?
Edit: by the way, what you’re repeating is what I jokingly call a “party line”. Especially if you look at the hard cold data. For example people often cite narrower spreads etc. but even with Interactive Brokers you can get MPID displayed on the NMS aggregated full depth order book and see who quotes how much and at what levels and the spreads don’t add up to the half of the myths people keep repeating. It’s easy to hide the real numbers in the _averages_ and various other statistics though.
The most vocal critics of HFT are very often previously practitioners who are upset when their strategies and models becomes obsolete are are outwitted by even more efficient operators.
Give any example of where you are going?
I will say RegNMS and NBBO regulations are actually preventing even further efficiencies. Dark pools and off exchange matching or internalization are complex topic that are easy to misunderstand. There are absolutely bad actors to be found in the system. This is true in any system. But on aggregate the system is continuously reducing costs and improving efficiency.
Intermediation costs are a friction on the real economy and capital markets and they will always exist, but on aggregate they are dropping for all users, institutional and retail.
What I am saying is that the thesis that the costs are dropping due to HFT style strategies has not been proven. Majority of the HFT tend to be market makers which tend to help with liquidity but not all are. I agree that liquidity helps offload or acquire large stakes.
As for NBBO and the effect a regulation has on markets you seem to be extremely US centric but if you go across the ocean and find out that Europe has no concept of NBBO at all, and that a retail person trading experience is equivalent for getting a different price on Amazon depending on which web browser they use you could imagine how that would make an average retail person feel.
Another example of the regulation is trying to move a large stake outside regular hours when NMS is suspended. Why do you think OTC block trades are pre-arranged at a fixed price? You can even look them up in relevant reporting facilities.
As for dark pools, ATS and internalizers there’s nothing difficult about them. I don’t think you’re doing anyone any favors by obfuscating an extremely simple concepts. Those market participants with their specific mechanics that are learnable. What’s complex is how to devise strategies and how to rely on the liquidity sources to get your desired fills and desired rates. You’re describing bread and butter of trading at an institutional level and the fact it seems difficult can maybe be attributed to the fact that it is opaque, doesn’t enjoy public communities, e.g. very little blog posts exist on the subject and the knowledge is sort of centralized to a specialized corners of the industry. But it is learnable without great difficulty if you have access to the resources.
What’s surprising to me is that a lot of people can talk about abstract concepts in computer science and then don’t connect facts that bridge into a separate discipline. As an example I could refer to consistency that people love to rave about but somehow forget the concept the moment capital market is introduced.
And yes, I’m familiar with the no trade theorem and relevant academic concepts but they are all models that often don’t translate to real life due to extremely limiting assumptions that don’t enjoy any connection to reality. I think it’s a problem with the community in this particular industry to take academic market models at face value. Often you will find that looking at a model carefully with assumptions that mirror real world, your performance gets better.
I agree it’s all quite understandable if people had the resources available. I would argue part of the reason it is opaque is the fears of operators that too much public attention can bring a lynch mob. This business can literally be one of the most profitable activities on the planet, but contrary to populist sentiment this is not evidence of wrong doing, it’s simply because it’s so automated and scalable and all electronic.
Back to ATS and dark liquidity, this isn’t as trivial to explain as you suggest, it requires understanding of flow toxicity and information theory to be understood completely. You must know that in markets, transactions can be beneficial to both parties as they have different objective functions and timescales.
What evidence can be presented that supports the idea HFT operators are extracting increasing costs on the system? I’m not aware of any.
What you’re failing to disclose is that there is an easy way out of solving the adverse selection problem. For example you could buy “uninformed flow”. I think that you could agree that for the touted sophistication of the field you would expect something … more sophisticated?
Again, don’t get me wrong. I love the field but I think it’s stagnant in certain aspects and I like to have a sober view of it.
It’s too late in the day for me today to attempt to write a coherent and accurate explanation of exactly why I say that. However I will say you don’t understand this isn’t a zero sum game and it’s an incorrect argument to claim HFT is an arms race in a game like a war, which is a negative sum game.
Your risk argument doesn’t hold water because liquidity is not the only risk. There are at least 20+ I can think of from the top of my head without even trying.
I’ll rest my case here.
HN isn’t the best forum for this type of ongoing discussion. Engage me on reddit if you like. u/alchemist1e9
Here’s an example of P&L dynamics:
At time t, your throw an unbiased coin. If it comes out heads your wealth multiplies by 0.6 with probability 0.5, or if it comes up tails, your wealth multiplies by 1.5 with probability 0.5.
Now you could simulate this process and take two averages. One is an ensemble average, averaging over many trajectories (of many participants) at a pre-defined time step t.
The other average is a time average (what happens to a a single trajectory picked at random over time). You may find the result interesting and close to what you’ve just described. This result is because of the process being non-ergodic; but it exhibits a few “winner takes all” and has nothing to do with the properties of the winners. It is a purely random property. An illuminating exercise. There are quite a few other results like this one from the field of stochastic processes that relate volatility bounds to your expected P&L, etc.
It’s unfortunate that calling out other’s bias is off limits in society too much as it’s often the primary reason people advocate their position.
HFT is not anything like what critics claim and those critics have severe biases.
https://news.ycombinator.com/newsguidelines.html
If you know more than others, that's great—please share some of what you know, so the rest of us can learn—or else don't post. Sneers and putdowns only make everything worse.
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
instead of order entry + continuous matching: order entry is allowed, but no matching (like pre-open phase)
then after a random period of time the auction algo runs on the entered orders
then repeat the entire thing every second
The strategies though are where the discovery is. There are a few strategies that are well known and still profitable but those are largely consolidated to the biggest firms. For everything else it’s a discovery process. And done strategies are only profitable for very short regimes.
I miss it sometimes too, but so much has been consolidated it’s largely a big firm world now.
More straightforward, heh, sure. But still damnably complicated. Which just goes to show how much money and how much engineering talent is invested in this world that these things are so taken for granted.
What did bother me, and was acknowledged by my coworkers, was how much top talent was being pulled away from productive tasks to essentially wank around. Not just technologists either, but all sorts of mathematicians and scientists were drawn to the flame.
We as a society have managed to allocate so many of the “best and brightest” to either fintech wankery or placing ads in front of eyeballs. It’s enough to make one want to give up on capitalism, except that everything else appears to be worse.
There is 0 social impact. That’s the downside of course - but hey, how many jobs out there are really having any kind of positive social impact ? Not 0, but close to it.
Intraday financial games are zero-sum. What HFTs gain, they leech away from mutual funds and pension funds and retail investors and market makers who operate over a longer horizon.
Regarding social impact, the world does have some demand for liquidity and price discovery. Providing those services is both essential and extremely difficult. It's definitely not the most social good I could be doing with my talents, but I think it's weakly positive.
Those things have driven a load of proprietary and open source tech that helps everyone else.
That depends on how you measure it.
Obviously this would mean that countries that previously relied on subsidised labour from those third world countries would have to start paying up. I'm not suggesting that it is a zero sum game however.
The problem with ads is not that they're useless, it's that it is an industry prone to scams and grift. (Because doing advertising right requires all sorts of actual science, and ain't nobody got time for that when there's money to be made.)
t. Worked for 20 years and adtech.
It's nothing to do with "we as a society". I'm a quant trader and know many others, and the vast majority are in the industry because we care about making money not some leftist save the world crap. Even if socialists managed to completely destroy the financial market, we'd just find another way to make money without trying to save the world (e.g. like the mostly corrupt officials in Russia and China when they were communist). "Society" can't change human nature; even mass brainwashing on the scale attempted by Maoist China failed.
I.e. the best and brightest will always work where they want to work, not where leftists like you want to "allocate" them.
Cut the fascist bullshit. Everyone can comment about how society should be run, even people that are very successful financially.
If you try to exclude greed from the design of your societal system, it will immediately fail. In large numbers, economics shows us that altruistic people wash out of the model and everyone operates in their own self-interest (greed).
> Cut the fascist bullshit. Everyone can comment about how society should be run, even people that are very successful financially.
It's not financial success that precludes someone from commenting on how society is run, it's blatant, deliberate anti-social actions. This is the foundation of a social contract. It's the same reason you'll get locked up if you go around punching people in the face.
Also, I don't think you know what "fascist" means.
It doesn’t seem like fascism to you because you’ve done some mental gymnastics to pretend you’re not just crushing intellectual opposition, but that’s all it is in the end. Op is not punching anyone in the face. He/she is just blatantly unsupportive of socialism.
Please, I want to know; what benefits do the actions of the op have for "societal planning"? The person he was replying to was at least honest in admitting that this behavior "served little social good".
Once again, I think you'd do well to actually read what fascism actually stood for. The propaganda of fascism had a lot more to say about nationalism, "shared history" and the good of corporations than it did to say about things being "anti-social" or even mentioning "selfish greed".
Largely, though I receive 1 or 2 job specs every week for start ups with the keywords 'hft' and 'low latency'. Admittedly there's going to be duplication there if you read them closely.
I think it's a bit of a myth that (ignoring FPGAs) that writing a low-latency software trading system is a time/cost expensive process. Anecdata = I worked at two firms where we did a rewrite from scratch with teams of 5-6 people and traded in the market within 3 months. I'd argue a senior dev that's been around the block a few times could achieve similar when you remove corporate politics, and bikeshedding over design.
The big firm part is paying for multiple quants at $200k++ to come up with strategies and historic market data access for trading models. Small firms are getting backing as long as the co-founders are 'ex-CxO from MegaCorp'.
This depends a lot on the complexity of the trading system and the trading venue specifics. A system to trade single stocks or futures can be built, certified and running in 3 months. A system for options market making will take a lot longer.
The big costs for small firms are historic data (if you don't have any), colo, distance to exchange, and number of connections.
From the number of job specs I see, it feels like the HFT/low latency market place is healthy enough that there are always new firms appearing. It's competitive, so it's hardly surprising that if someone has new ideas they'll find a backer.
HFTs are definitely playing a completely different game. I was reading about the exchange architectures and how things are actually wired. I'm getting my data from SIPs while HTFs are directly connected to the exchanges [1]. I'm transacting in seconds and they, like you said, are transacting in microseconds, so there is no comparison. Which, in a way is actually nice in that I'm not really competing with them. Or, maybe I am but I can still make some money. haha.
Cheers and thanks for the awesome comment!
[1] https://www.researchgate.net/figure/Latencies-in-the-Electro...
In the old days before HFT, you weren't sure you'd get the best price. You'd have to rely on a broker to make sure that happens, but as a retail trader you generally got a worse price/out of date price.
Nowadays with HFT you can get pretty much the best price anywhere. Those <1ms HFT traders make that happen. It's the efficient market hypothesis in effect, made possible by HFT.
Now is it pointless to shave off even more ns in the all out war to grab a piece of the order flow? For retail and institutional investors, at a certain point, yes it's completely pointless.
But it's also just pure capitalism at work. HFT firms compete against each other, and the competition is about speed to provide the best price and volume. If you try to regulate with something like enforced delays, then what do you compete on instead?
Capitalism necessitates efficient markets (and efficient markets necessitate HFTs) so any criticism of HFTs is a direct criticism of capitalism as well. I mean this isn't really a problem that's specific to HFTs -- there are just a lot of jobs that we can perceive as providing no value or even negative value (jobs that are possible specifically due to capitalism e.g. payday loans, 2008 style trading, certain scams or predatory practices etc.)
I think we can all agree that this is a flaw, and we're not criticizing capitalism to replace it with something else, but rather just recognizing this as a problem. At the end of the day, a useless job is a useless job even if it exists solely because of capitalism.
It feels like there would be a lot of incentive to keep an inefficient system in place in order to keep the grift going, and a lot of convenient excuses available for why something better isn't possible.
People are saying this because, HFT sounds similar to 'crypto mining'. That's people with best infrastructure, the 'big-guys' -- win. While leaving out the retail investors as broiler chicken, pumped with 'drugs' (by influencers) to spend more on imaginary assets, so that they can be used for 'food' by these 'big-guys'.
There are different influencers for retail investors vs crypto. In retail investing there are promises of 'retirement paradise', actual tax deductions, the Jim Cramer-like people (at least what I heard in US)
For crypto investing the influencer are different, the geography is wider. A promise to participate in markets if you do not live the country that has adopted US/UK-based financial services.
- - - By the way, I think the markets will still have liquidity if there is a rule to wait, for say, 30 min before a stock that was just recently bought -- can be sold (unless by a clear fat finger mistake)
This rule will cause the HFTs to stop existing in the current form.
Do any regular pedestrian middle class retail investors EVER actually earn enough to retire on? Or do anything meaningful with? If so.. what is THEIR secret sauce, since it's not HFT...
a)Members of political elite that get insider trading stock tips. (illegal of course). The number of folks in usa congress and senate that become 'very lucky' investors after they join the rank, has to be amazing
b)Lucky
c) everybody else -- that looses.
Overtime, I would say last 30 years, the amount of 'influencing' retail investors to trap them into unreasonable actions had gradually increased.
So the percentage of folks going becoming the victims of the charade, will become higher.
Certainly if you concentrate on ( b ) you create the plausible deniability defense for the manipulators
You are profiting off workers as a middle man in the economy by doing HFT, and trying the justify it by some vague concept of the "correct price". You are producing nothing of value, merely taking away value before someone else notices it is there.
Our entire economy runs on "middlemen". Convenience has huge value to most people.
It's a bit like stock brokers - and why wouldn't we want stock brokers to operate at drastically faster-than-human timescales, because we all know the value of a company changes every nanosecond! And "flash crashes" create opportunities for investors to make huge amounts of money!
And just as crypto has poured money into GPU companies, HFT has poured money into networking companies.
Short of creating the great firewall or helping governments slurp up all the traffic on the internet, what could be a more beneficial application of network technology?
HFT (like finance in general) has also gobbled up lots of tech grads, making it easier to find jobs at tech companies.
Now when I say the same thing about index funds people get all huffy
Regarding liquidity, I imagine it's possible that the index fund might allow you to withdraw money faster than you could sell on your own, but realistically I kind of doubt it.
Likewise with crypto. Personally I don't think that if you're calling an API over an Internet it matters if your trading bot is written in go or python (mine was in python). Use the language you're most comfortable in. The network and trade submission/execution at your broker will be 10x slower than your bot anyway. Unless the size of your operation approaches the size where you can get direct market access which seems to be reserved for big forms only.
That's just it. The whole premise is pretty absurd. The market, the actors, everything. It's so far removed from literally anything remotely human. It's the financial equivalent of an infinite sea of AI bots producing CVs and research papers which are only being evaluated and read by other bots.
If you step away from it all for a second, what the hell is the endgame of this whole hustle.
Anytime you create technology that sufficiently replicates the creators, you end up with the spirit of the creator embodied in the technology. So, of course digital brains are going to do weird things like crossword puzzles and sodoku at scale, because it's the same kind of useless shit we do to entertain ourselves.
You get into this conversation whenever you dive hard into cyberpunk, which is so many, many things. Even the Internet itself started out that way. The endgame started as a game, and it will end up being a game, played by our creations as odd mirrors of their creators.
I think there's a lot of people who subscribe to doing the same thing to save the planet, of which I have a keen interest. Solarpunk is the name of that movement, and it also has similar crazy ideas.
We're inventing digital brains. It's literally an architecture designed to be removed from being anything remotely human as it's a mimic or replacement technology for intellectual capacity.
If you really take a step back, the endgame is crazier than just bots producing content merely for other bots to consume (which basically describes the vast majority of scientific papers these days too, ironically). When you take that absurdity and multiply it by tens of thousands in terms of efficiency, the whole system we're building looks WILD and almost inconceivably strange to the way we do things now.
I know this is only tangentially related, so I appreciate your understanding that I already understood that and wrote this anyways. :)
Is it basically "do what MFT does, but faster", or is there any specific advantage like getting into an order queue with priority?
"This aspect, the platform itself, seems to be often overlooked in most discussions. Many conversations revolve around strategies (mean reversion, trend following, linear regression, etc.), and backtesting, without fully addressing the practical mechanics or logistics of strategy implementation, particularly in the context of live, intraday trading."
I'm glad you had fun, OP, but also I think I can shed some light on why most people discuss strategy.
Trading is a perfect storm of ridiculously high tech, ridiculously complicated, ridiculously regulated (Not over-regulated, mind you, this isn't a value judgement. But the amount of regulation is extremely high.), and ridiculously competitive.
But that said, it's the last bit that drives it all. Since it's so competitive, even though building an order entry system, and a risk system, and a position-tracking system, etc is a huge accomplishment (again, congrats OP!), it's table-stakes to even dip your toes in the pool here. Trading shops can attract top talent and robust, bespoke trading systems are basically cost of entry.
So people talk about strategy because everyone already has the table-stakes stuff and are now trying to make money with it.
It doesn't help, too, that lots of market participants aren't even playing the same game. In HFT, we operated on trades with alphas that lasted a few seconds, where races to entry/exit were battled in shaving nanoseconds off FPGAs being able to shoot out orders and microseconds off wireless networks flying market data around new jersey. Meanwhile, banks are more concerned with elections and geopolitics than they are about the weather in Carteret. (Rain = no microwave network for the day). And then there's a million strategies in the middle with alphas that last from hours to weeks.
So it makes it really hard to even speak the same language to each other when talking in common forums.
It's a fun world. I miss it sometimes.