News for hackers. Stuff that matters.
Re:A detailed data-centric exposé on how the market is rigge... by bluejekyll on Wednesday November 18, 2015 @11:06PM (#10592674)
If this is legit, what amazes me is the time! Network hops can account for approx 100microseconds, which means that these HFT trades would not only have to be in the same datacenter, but within no more than 2-3 hops of the exchange device. Then on top of that, it's making its trade decision within 500 microseconds.

I need to sit down to calculate this, but that is guaranteed to mean that they are reading from RAM and not disk. Could the entire stock market fit in a single hashmap in memory?

That's just some awesome hardware and super efficient software.

Edit:typos

Re:A detailed data-centric exposé on how the market is rigge... by gruez on Wednesday November 18, 2015 @11:20PM (#10592723)
>Could the entire stock market fit in a single hashmap in memory?

I don't see why not. You also don't have to put the entire market onto one machine - you can shard the market across multiple servers, with one server handling each symbol.

Re:A detailed data-centric exposé on how the market is rigge... by bluejekyll on Wednesday November 18, 2015 @11:35PM (#10592766)
> shard the market across multiple servers

Sure, but now you've added a router, consistent hash or the like, to the picture, which is either another hop or a custom physical rack instead of a traditional ToR. It's not impossible at all, it's just impressive that they do all of this in 1ms!

Re:A detailed data-centric exposé on how the market is rigge... by JumpCrisscross on Thursday November 19, 2015 @12:35AM (#10592939)
We'd burn logic to chips once a week. That was back in 2011, however.
Re:A detailed data-centric exposé on how the market is rigge... by gruez on Thursday November 19, 2015 @12:07AM (#10592866)
>Sure, but now you've added a router, consistent hash or the like, to the picture

not necessarily. If the HFT companies are using some sort of secret internal API (very possible as they like to shave milliseconds off their execution time), it's possible that they are talking directly to relevant server, bypassing any loadbalancers or routers.

Re:A detailed data-centric exposé on how the market is rigge... by omgtehlion on Thursday November 19, 2015 @07:42AM (#10594406)
On a regulated exchages you cannot bypass official gateway. Though you do not need to. The only thing you want is to be faster than other traders.
Re:A detailed data-centric exposé on how the market is rigge... by sabarn01 on Thursday November 19, 2015 @12:25AM (#10592913)
It wouldn't surprise me we were using fpgas to do fix decoding and we were relatively naive.
Re:A detailed data-centric exposé on how the market is rigge... by nostrademons on Wednesday November 18, 2015 @11:56PM (#10592835)
Pretty easily. With an efficient data representation, the entire stock market could fit in L3 cache.
Re:A detailed data-centric exposé on how the market is rigge... by fleitz on Wednesday November 18, 2015 @11:41PM (#10592786)
Yes, Google "LMAX Architecture", only the tape is written to disk.

The order book itself is tiny and there are only ~20,000 stocks, and you only have to fit one stock per computer.

Re:A detailed data-centric exposé on how the market is rigge... by ximeng on Wednesday November 18, 2015 @11:57PM (#10592840)