News for hackers. Stuff that matters.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by lordnacho on Sunday March 10, 2019 @05:36PM (#19354701)
I was working in HFT myself, though in Europe, and I asked around about this. Specifically WTF do they do with this line from Chicago to New York that's so insanely important?

The US market has this interesting phenomenon that the futures trade in Chicago, but the single stocks trade in New York. For those who are not familiar, this means you can make a bet on the S&P 500 index as a whole in Chicago by buying or selling futures. But if you wanted to replicate the basket, you would go to New York.

Naturally there's a relationship between the two. If the futures are going down, the prices of all the stocks needs to be adjusted. And if you look at stock prices, it turns out quite a lot of any stock price is simply exposure to the market at large. So you really want to know as soon as possible if the market is moving.

Now you might have thought that the index is the index of shares, which is in NYC, so why does it matter what Chicago thinks? Well actually the tail wags the dog in this case. The futures are how people express a view on which way the index is going, and you want to know that view immediately.

How does it actually work? Variants on "if the market goes down, pull all your bid orders". Basically if things are fine you are happy to look at queue position and imbalance as an indicator of whether you want to leave orders to capture spread with. But the moment there's an indication from Chicago that the market is dipping, you forget that and pull your orders. Or if you're really fast dump a sell order on the guy behind you in the queue.

There's a huge amount of stock trading in NY each day, so it's worth your while to have this line that you use to send the warnings down.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by chirau on Sunday March 10, 2019 @06:23PM (#19355029)
It's basically arbitrage. Information and time difference arbitrage. Different exchanges on the East Coast, because of their different locations receive trade info at different times. This is what they are trying to capitalize on. And also, the obvious race to first receiver of information from Chicago.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by GusRuss89 on Monday March 11, 2019 @12:48AM (#19356759)
Do the companies that do this make any claims about how they benefit society? To me, stock traders in general seem a departure from the idea that money is a result of providing value to society, but arbitrageurs seem even further detached. I'd be interested to hear if there are arguments I haven't thought of.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by URSpider94 on Monday March 11, 2019 @03:37AM (#19357290)
High frequency traders create liquidity. Arbitrage in general helps ensure that prices are uniform across geographies and across linked securities (e.g. options are priced appropriately vs. the underlying asset). You can argue that there is no incremental value in going to shorter and shorter timescales, but fundamentally this mechanism is how information moves around the economy.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by tcbawo on Monday March 11, 2019 @08:46AM (#19358515)
There used to be less sensitivity to the receive-side time by exchanges, but in many cases there had been an emphasis on fairness and high res timestamps (they sell colocation services). Now, many times the first through the post wins. This creates a constant arms race and narrows the field to a handful of competitors. Many companies that survived in the fat tail can no longer do so.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by lordnacho on Monday March 11, 2019 @03:21AM (#19357252)
A guy is willing to buy some shares at 101 on exchange A. Another guy is willing to sell the same shares at exchange B for 99. Neither of them has the budget to connect to both exchanges.

HFT has connections to both and can see the cross. Everyone is better off.

This might be different in the US due to NMS, but HFT strategies are all pretty similar: they can provide liquidity better if they are fast, and other people can take advantage of that.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by MuffinFlavored on Sunday March 10, 2019 @07:49PM (#19355551)
I'd be curious to hear some statistics on how much money a ~10ms difference in time of information received can actually make.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by kasey_junk on Sunday March 10, 2019 @08:09PM (#19355644)
I don’t have any statistics but I have an anecdote.

I built a trading platform ~6 years ago that had a 100 microsecond tick to trade (a term that means from the time a market update hits your nic to the time an order leaves your nic).

We knew then that we were too slow for large swaths of the HFT space & wouldn’t even attempt those trades.

I haven’t kept up with the industry but I’d guess tick to trade times at the speed edge are in the low hundreds of nanos now. In an environment like that a 1ms time improvement is out of the capabilities of your software/hardware engineers and can only happen at the network.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by aldanor on Sunday March 10, 2019 @09:18PM (#19355935)
If we’re really talking “at the speed edge”, low hundreds of nanos would be already considered too slow these days :)
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by chirau on Sunday March 10, 2019 @08:11PM (#19355660)
Hold my beer ...

https://www.youtube.com/watch?v=L5cZaIZ5bWc

And that was more than 6 years ago, one ticker symbol only. You can imagine now.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by titanomachy on Sunday March 10, 2019 @10:16PM (#19356192)
Holy shit.

That gave me a much better sense of scale than hearing "a trade takes 200ns" or whatever. The engineering that goes into making this possible must be just insane.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by SpicyP on Sunday March 10, 2019 @08:47PM (#19355821)
If you’re an options market maker and you are 10ms slower, you legitimately will be obsolete in the market. You will be arbitraged against when stocks move and you are slow to update quotes. This happens all the time, and it makes it so that you have to quote so wide that you aren’t competitive. Milliseconds are the difference from being competitive and going broke in the industry.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by lordnacho on Monday March 11, 2019 @06:40AM (#19357934)
10ms is way too long. Think about 3D graphics drawn at 60fps. That's 16ms you get for each frame. But 3D graphics calculates a LOT of stuff for your whole game in that time.

In trading you're not calculating nearly as much stuff.

In trading it's probably very sharp, the line between making and losing money. You want to be under 100us for sure. That's micros. Depends on what you're doing of course but 10ms is OOM to slow.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by shagie on Sunday March 10, 2019 @07:28PM (#19355432)
Following up on this... The blog post Islands at the Speed of Light ( http://www.bldgblog.com/2011/03/islands-at-the-speed-of-ligh... ) with a "global map of 'optimal intermediate locations between trading centers,' based on the earth’s geometry and the speed of light"

There's a similar article at When The Speed Of Light Is Too Slow: Trading at the Edge ( http://www.kurzweilai.net/when-the-speed-of-light-is-too-slo... )

The paper that both of these articles are based on: Relativistic statistical arbitrage https://www.alexwg.org/publications/PhysRevE_82-056104.pdf which opens with:

> Recent advances in high-frequency financial trading have brought typical trading latencies below 500 􏰒µs􏰃, at which point light propagation delays due to geographically separated information sources become relevant for trading strategies and coordination 􏰀e.g., it takes 67 ms, over 100 times longer, for light to travel between antipodal points along the Earth’s surface􏰁. Moreover, as trading times continue to decrease in coming years 􏰀e.g., latencies in the microseconds are already being targeted by traders, this feature will become even more pronounced.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by dmurray on Sunday March 10, 2019 @07:39PM (#19355487)
I've seen this paper before, or something like it. It's incredible to me that anyone would publish this. It's like reading a paper on astrology or dowsing, written in the style of a physics paper.

You don't get any benefit from putting your trading system half way between the two exchanges, you colocate it with one or the other (or both). Likewise if there are three exchanges to trade on, you don't build a datacentre at the circumcentre of the triangle or whatever voodoo geometry nonsense. You colocate at each of the three and buy/rent/build a line for each of the three point-to-point connections. Source: worked in HFT for more than 15 minutes.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by loxias on Monday March 11, 2019 @02:09AM (#19357027)
I also know one of the authors quite well, long term friend. He is nothing short of brilliant but also has a penchant for enjoying being a "subtle troll". While he does have a serious point/observation/discovery in the paper, I assure you, he's well aware of the "funny" or "goofy" aspect of the impracticability and laughing about it has been a source of great late night entertainment between us.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by mattrp on Monday March 11, 2019 @12:50AM (#19356766)
I know the author well and he would likely conced to you some of the practical points you’re making about collocation and market microstructure. However he wrote the paper to illustrate how global compute networks could someday collaborate using the techniques described to facilitate AI. Hft is a early form of AI so if that’s an area of interest for you maybe take another look at that paper.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by thechao on Sunday March 10, 2019 @08:13PM (#19355669)
I've heard the (facetious) comment, before, that if we ever do discover a faster-than-light theory it'll be because of HFT, not academia.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by BonesJustice on Sunday March 10, 2019 @05:57PM (#19354850)
There’s also more straightforward reasons to want a microwave connection from NYC to Chicago. Say you want to trade off news signals: you want to relay the headlines (from Bloomberg, DJ, etc.) to your colo sites as quickly as possible.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by pgwhalen on Sunday March 10, 2019 @06:12PM (#19354953)
Can anyone in the industry comment on whether this kind of strategy really exists? Surely any automated, news-based strategy's bottleneck is not due to sub-microsecond latencies stemming from microwave tower placement. I've always assumed that the strategies that require microwave networks are using market data and only market data as a primary input.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by BonesJustice on Sunday March 10, 2019 @06:17PM (#19354990)
I can confirm first-hand that this is a real thing, and that if there’s significant movement immediately following a headline, you likely won’t end up with more than table crumbs if you’re not one of the first two or three players to respond.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by dmix on Sunday March 10, 2019 @06:42PM (#19355168)
Interesting. I’m curious, can you give an example of what kind of news? Some earnings reports or acquisition or something?

And are the reactions automated based on some NLP? Milliseconds wouldn’t matter if a human has to still parse it, correct?

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by darawk on Sunday March 10, 2019 @06:57PM (#19355274)
Yes, NLP, and sometimes structured data released by special services that receive the news alerts early, process them into a structured form, and then make them available at the same moment that the normal services do.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by ineedasername on Sunday March 10, 2019 @10:11PM (#19356176)
Yep, NLP. As a grad student in comp linguistics, I was surprised to find out that financial news was an enourmous application space for NLP, for exactly this sort of reason. Including the automation of content creation. Much financial news isn't written by a human, and hasn't been for a long while. It's too slow. Faster content creation equals faster actionable data equals huge advantage.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by BonesJustice on Sunday March 10, 2019 @07:26PM (#19355425)
NLP is a possibility, but bear in mind that HFT is all about optimization. The confidence levels desired can vary significantly based on what you intend to do with the information.

Acting on M&A news by taking a more aggressive position generally requires more confidence than, say, backing off based on ‘bad’ news. In the first case, you want to be pretty sure of the headline before taking action. In the latter, the mere presence of words like ‘fraud’ and ‘investigation’ might be enough to play it safe and say, “we’re going to back off trading this underlying until a human has a chance to look at this.”

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by shitgoose on Sunday March 10, 2019 @06:50PM (#19355228)
or you misread the headline, price reverts and you end up with losing position. i am genuinely curious, how feasible this headline trading is.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by pmart123 on Sunday March 10, 2019 @08:04PM (#19355623)
I used to be in this business. It’s a good practice to think of everything in terms of probabilities alongside capacity/liquidity. If an all cash merger hits the news, there’s a race lift prevailing offers on the one stock as there is a much better expectation of value due to the news.

Now let’s say there’s negative news on the Chinese economy. Someone could be running an algorithm that sells down companies with Chinese exposure. But, inherently, there likely is a much lower probability that this news is uniform or material across companies with China exposure. A strategy like this likely would get put on small positions and get in and out quickly as it could get run over depending on how humans process the information and make bigger discretionary bets.

There definitely is a risk too all of this. Roughly 10 years ago, somehow an old united Airlines bankruptcy article hit the wire and the stock sold off hard. Eventually, United released a statement saying the article was innacurate. This isn’t typically the type of strategy you would make large, single bets with, or hold the positions over a long period.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by BonesJustice on Sunday March 10, 2019 @07:12PM (#19355356)
There’s two sides to it: you might decide to take a more aggressive position, but you might also use the headlines as a signal to back off (and stop or scale back trading of securities for the affected underlying, at least temporarily).

One tends to carry more serious consequences when you’re wrong. Accordingly, the ‘take’ path generally has more checks, and more opportunities to abort.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by TuringNYC on Sunday March 10, 2019 @07:31PM (#19355440)
>> or you misread the headline, price reverts and you end up with losing position.

Yes, you might misread here or there. But this is a game of statistics and the real win/loss is whether you read correctly more often this read incorrectly. You want your net wins to be positive.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by raviolo on Sunday March 10, 2019 @10:57PM (#19356328)
Quite a few players are trading on economic releases (eg non-farm payrolls, ADP, ISM, and so on) in this fashion. It’s often much easier to feed to algo because the release is a number (or set of numbers) vs some free-form-text thing. Competition is at a sub-micro latencies today.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by integrate-this on Sunday March 10, 2019 @08:28PM (#19355747)
Yes, this exists.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by nodesocket on Sunday March 10, 2019 @06:05PM (#19354904)
Thanks for this explanation. I previously thought the distance fight was over being closest to the NASDAQ co-location building in New York, not connecting Chicago and New York.

I am still not clear why microwave is faster than fiber though. Is it simply that the fiber lines are multi-tenant, so there is not a direct hop?

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by zkms on Sunday March 10, 2019 @06:57PM (#19355276)
Light in glass is slower than light (or radio waves) in air. Also where they're unable to put point-to-point microwave towers, like the Atlantic ocean, they use HF/shortwave (as in 3-30MHz stuff) radio, because frequencies in that range bounce off the ionosphere. And apparently, the added lag from the increased path length is less than the added lag from glass's optical index, so HF wins out. HF is, however, very noisy and the bandwidth (amount of bulk bits per second you can send) is not too high (even though the end-to-end latency for a single given bit can be very low).

It's not pure HF -- there's a fibre link alongside the HF link where stuff like protocol negotiations, encryption keys, channel sounding, feedback on data received and RF conditions, and agreement on what kind of modulation/coding schemes to use for upcoming data (and what transmitted bits are to mean in terms of higher-layer protocol (something about price informations? I don't know much about HFT)).

Likewise, the availability of GPS makes time/frequency synchronisation easier (and combined with sounding signals and feedback over fibre), which very likely allows for eliminating latency-inducing complexity in the receiver.

It's a far cry from the traditional data-over-HF scenario, which is assumed only to be relevant when other (higher bandwidth) links such as fibre or satellite or point-to-point microwave are unavailable, so everything must be carried out over the RF channel (look at ALE).

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by packet_nerd on Sunday March 10, 2019 @06:12PM (#19354949)
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by Scoundreller on Sunday March 10, 2019 @08:02PM (#19355612)
Also consider the path: you can pretty much do a great-circle with radio.

A cable is not going to be direct. If it's above ground, it needs some slack on the line to account for pole movement (you don't want your fibre holding two poles together...).

You'll need more slack for your splices to be done at surface level during construction. Splice trays every x meters will add more distance.

Sometimes the poles are overloaded and you need to run down and bury it for a while.

Your tunnels are never really straight. You need to follow existing easements, or get your own. When you do bury, you need to go around various physical obstructions, like rivers, buildings, cities, parks.

And if you want redundancy against back-hoe fade, your latency in those situations will shut you down, perhaps erasing your profits when you're down and your competitors are up.

The cost of running your own line point-to-point would be be monstrous, though I guess you could build in some latency and sell access to non-HFTers.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by jmole on Sunday March 10, 2019 @06:09PM (#19354926)
the index of refraction of a material is essentially a ratio to the speed of light in the material, where free space is 1. Glass, being around 1.5, has a lower speed of light.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by lukateake on Sunday March 10, 2019 @06:13PM (#19354963)
So that's why my diamond optic cables never achieved product-market fit! http://hyperphysics.phy-astr.gsu.edu/hbase/Tables/indrf.html
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by hyeonwho4 on Sunday March 10, 2019 @06:36PM (#19355120)
That, and how the fick do you machine diamond... regular glass fiber optics needs to be cut to a smooth surface and fused together. The devices to do this use diamond-coated blades.

Oh, and diamond has a large birefringence due to its crystal structure, which would cause temporal smear in signals.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by 6nf on Sunday March 10, 2019 @08:06PM (#19355630)
Fiber is about 0.6c and the cables are much longer than direct line of sight.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by wereHamster on Sunday March 10, 2019 @06:14PM (#19354969)
Microwave signals travel from the transmitter to the receiver in a straight line. Whereas light bounces off the walls of the fibre optics cable, which increases the distance. Furthermore, the material that fibre optics are made out of attenuates the signal much stronger than air, so you have to place more repeaters and electronics in between your endpoints.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by packet_nerd on Sunday March 10, 2019 @06:26PM (#19355050)
> light bounces off the walls of the fibre optics cable

This is true of multimode fibre, but not single mode. Long haul communications cables are single mode.

https://en.wikipedia.org/wiki/Single-mode_optical_fiber

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by Denvercoder9 on Sunday March 10, 2019 @07:37PM (#19355480)
Even for multimode fiber it's a huge simplification of the actual physics. The actual cause of the speed difference, for both single and multimode fibers, is the index of refraction of glass.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by mattrp on Monday March 11, 2019 @12:43AM (#19356732)
Look up refractory index. That’s all you need to know. The above is meaningless in this context.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by consz on Sunday March 10, 2019 @06:11PM (#19354941)
Light signals in an optic fiber bounce off the walls (repeatedly, frequently) of the cable they're in -- the actual physical path the light is taking in this case is approx. 1.5x the path of the cable itself.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by hyeonwho4 on Sunday March 10, 2019 @06:30PM (#19355077)
That's not true. The typical total reflection angle in single mode fiber optic cable is less than 4 degrees. The refractice index of glass, however, is about 1.45. So almost all the slowdown is due to the glass directly slowing down the signal.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by madengr on Sunday March 10, 2019 @07:10PM (#19355339)
Though you can still get a group delay (slower than free space) in vacuum filled RF waveguide. When you break down the single mode, it appears to be zig-zagging through the waveguide (bouncing off the walls), and slows down as cutoff is approached. Like in fiber, the modal dispersion is separate from the material dispersion.

AT&T was going to build out a cross country, single mode, circular waveguide network in the 60’s, but was obsoleted due to fiber. Maybe that’s what the HFT ought to due, or maybe air-line coax.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by kazinator on Sunday March 10, 2019 @06:21PM (#19355021)
I think the distance is less of an issue than the temporal smear that the bouncing causes. A clean square pulse turns into a blob.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by hyeonwho4 on Sunday March 10, 2019 @06:34PM (#19355104)
Temporal smear is caused by the varying refeactive index of glass with wavelength and the fact that a short time pulse must have finite (large) spectral width (by the uncertainty principle), not by the bouncing.
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by kazinator on Sunday March 10, 2019 @06:46PM (#19355206)
So, the different path lengths implied by internal reflections don't contribute anything to the smear?

By the way, the uncertainty principle doesn't say we can't have a narrow pulse. Frequency versus time can be used as an analogy to help explain the uncertainty principle. If we know a precise frequency, the signal is poorly localized in the time domain. If a signal is narrowly localized in the time domain, it is poorly localized in the frequency domain. The uncertainty principle is something else, though: it revolves around not being able to make certain related measurements simultaneously, at the quantum level. We can be quite certain about the frequency content of a transient pulse and its exact temporal shape, if it's not something from the quantum domain.

Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by rhizome on Sunday March 10, 2019 @07:58PM (#19355589)
Is this why people move to Pittsburgh and Cleveland?
Re:A Gazillion-Dollar Standoff over Two High-Frequency Tradi... by tcbawo on Monday March 11, 2019 @08:52AM (#19358551)
The L1 repeaters along the path add very little latency and require no real presence.