News for hackers. Stuff that matters.
Re:Trading bots are running wild on cryptocurrency exchanges by phildaian on Monday April 15, 2019 @06:07PM (#19669349)
Author of the paper here. I justified our use of the word "frontrunning" on a Twitter thread: https://twitter.com/phildaian/status/1116698738343460864

There are a few different points here. Firstly, there are plenty of unambiguous definitions of frontrunning that are accepted by the scientific community that this blatantly does fall under; see the Clark SoK we cite in our paper for this in the blockchain space, or the definition Wikipedia uses, which was from an economics article by Khan & Lu (https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1140694): "trading by some parties in advance of large trades by other parties, in anticipation of profiting from the price movement that follows the large trade."

The second point here is even if you want to restrict the definition to require privileged information, privilege in p2p networks is not binary: there are some nodes that are better peered on the topology, have better mining equipment, etc. who can use these privileges to extract market rent, harming the security of a blockchain.

This is a complex and dense paper that took 8 people 18 months to produce, and I recommend you read it with an open mind, rather than having a gut reaction to whether we use your preferred definition of "frontrunning". I also recommend reading the primary source material directly, or at least my summary on Twitter - https://twitter.com/phildaian/status/1116155236433956865

Happy to answer any questions!

Re:Trading bots are running wild on cryptocurrency exchanges by tptacek on Monday April 15, 2019 @07:20PM (#19669746)
The definition implied by Khan & Lu is incoherent. It refers back to Larry Harris (Trading & Exchanges most definitely does not define frontrunning this way!) to suggest that, in attempting to execute trades for large blocks, traders might expose their orders and "create an opportunity for frontrunning". But that's basically the fundamental challenge of shopping a large block; it implies that essentially all of institutional trading is "about" frontrunning. That can't be right.

But if 51% of Wikipedians believe otherwise, who am I to argue?

I read the whole paper before I commented, but really wasn't remarking on the paper so much as this subthread. If I was going to pick nits, it'd be with the title of the paper and the allusion to _Flash Boys_, which is just a terrible book.

Late edit:

If you want to bring this back to computer science, I think it's helpful to think about this stuff in terms of the CAP theorem; according to our science, the opportunity to arbitrage latency in a continuous market basically has to exist.

Re:Trading bots are running wild on cryptocurrency exchanges by tanderson92 on Monday April 15, 2019 @08:23PM (#19670038)
FWIW, I have a copy of Harris' book on my desk right now and every explicit citation of Harris in Khan & Lu's paper appears to be accurate--supported by the text in Harris (Chapter 11).

edit: > But that's basically the fundamental challenge of shopping a large block; it implies that essentially all of institutional trading is "about" frontrunning. That can't be right.

That doesn't make the Khan & Lu definition wrong, just that there is a lot of "front running" that occurs. Larry Harris even explicitly mentions this point:

"When large traders recognize that they cannot trade as well as a professional trader can, they commonly hire a professional to help them trade. They may hire a block broker to act as their agent, or they may ask a block dealer to facilitate their trades. Block dealers who facilitate their customers' trade trade their blocks at uniform prices. The dealers then try to profit by trading the block in the market at a better average price. In a sense, block traders are front runners whom large traders hire to help them solve their trading problems"

This is related to the idea of sunshine trader discussed earlier in the chapter before that blockquote.

Re:Trading bots are running wild on cryptocurrency exchanges by tptacek on Monday April 15, 2019 @09:55PM (#19670506)
I have to concede you're right, and that the distinction that Harris draws is not between front-running and "jellyfishing" or whatever we want to call it, but between legal front-running and illegal front-running. Here's Harris's standalone definition, which is what I remembered:

Front Running: Front running occurs when a broker improperly allows one order to trade ahead of another. The order that goes first usually profits from the price impact of the following order. Front runners hurt the traders whose orders they front-run because they take liquidity that the front-running traders otherwise would have taken. These orders then fill at worse prices than those tat which they would have filled.

Front running is most common when a broker holds a large order that will likely move the market. The broker then trades for his own account first, or he tips off a confederate who does the front running.

Front running also hurts the brokers who represent the orders that are front-run. Broker clients who pay close attention to how well their brokers perform will discover that their brokers who knowingly or unknowingly allow others to trade in front of their orders do not trade effectively on their behalf. When their poor performance becomes apparent, the clients often direct their orders to other brokers. Firms that employ brokers must therefore be vigilant to ensure that their brokers do not cheat their clients and thereby lose business for the firm.

This seems pretty cut-and dried.

But, a little later in the book, there's a whole little subsection on "Legal Front-Running By An Observant Trader", ending with "[the front-runner's] trading is legal. Her profits come from recognizing [the broker's] shortcomings as a broker, and from noting that [the broker's] clients tend to split their orders. She is a profitable trader because she is observant and because she acts quickly on her information."

I fully concede this point.

It's of course important to retain the distinction between legitimate front-running and FINRA-style illegal front-running.

Re:Trading bots are running wild on cryptocurrency exchanges by tanderson92 on Monday April 15, 2019 @10:23PM (#19670640)
In fairness, it's a very Harris-y kind of chapter which cuts against how you might want to understand or talk about front-running with people outside the industry. (off topic: having met the guy I'm not at all surprised by how he constructs this chapter or the book in general -- which I love).
Re:Trading bots are running wild on cryptocurrency exchanges by pvg on Tuesday April 16, 2019 @03:09PM (#19676379)
In the dystopian timeline in the movie, they find The Citation's lifeless, sundered husk at the bottom of the ruins of what was once a massive thread. About DNSSEC.
Re:Trading bots are running wild on cryptocurrency exchanges by tptacek on Tuesday April 16, 2019 @04:29PM (#19677024)
OK I give up what is The Citation a reference to
Re:Trading bots are running wild on cryptocurrency exchanges by scottlocklin on Monday April 15, 2019 @06:15PM (#19669394)
It's not a "preferred definition." Your definition of frontrunning is simply incorrect.

FWIIW Michael Lewis is considered to be a serial fabulist by most people on Wall Street; his "flash boys" was a particularly egregious example of this, being a marketing effort for IEX, which was funded by friends of Michael Lewis.

Re:Trading bots are running wild on cryptocurrency exchanges by phildaian on Monday April 15, 2019 @06:22PM (#19669428)
I linked you to several sources for my definitions and you haven't commented on any of them, so no idea how you could reach that conclusion. I've explained to you that even with your restrictive definition, our paper covers analogous behavior in DEXes.

Yes, it's very surprising, no, shocking, that people on Wall St. are reluctant to have behaviors like this called out publicly....

I am not Michael Lewis and cannot speak to the accuracy of his facts; our work is a scientific paper backed by a lot of data currently undergoing peer review. The views on HFT and market design in the paper are similarly informed by cited work that is quite extensive in its data analysis, and peer reviewed. I recommend you read it!

Re:Trading bots are running wild on cryptocurrency exchanges by tptacek on Monday April 15, 2019 @07:23PM (#19669763)
Help me understand what you're trying to say with that middle paragraph, about how "shocking" it is that people on Wall St. are reluctant to have behavior called out? To what behavior are you referring?

I'll add: on Twitter you noted that virtually every HFT trader, lawyer, and quant pushed back (either to your definition or to your allusion to Lewis's unfortunate book). You seem to think that's added credibility for your argument. Isn't a near-universal rejection of that argument from subject matter experts sort of damning for the position you're trying to take?

Re:Trading bots are running wild on cryptocurrency exchanges by scottlocklin on Monday April 15, 2019 @08:47PM (#19670160)
You pointed out no such thing, and regardless; no such bona fide definition of "front running" exists. You got it wrong: admitting your error would be the honest thing to do. It's OK to be wrong; that's part of science and rational enquiry. It's not OK to not admit that you're wrong.

FWIIW I'm not on Wall Street; I work in tech. Wrong again.

Your results are both trivial and basically linkbait. Everyone who knows what a time priority order book is knows there are stale orders on DEX exchanges. The fact that people are arbing them is markets doing what they're supposed to do; creating one price across disconnected markets. Anyone who knows what they're doing and uses a DEX knows this; DEXes are for people who for whatever reason can't use central exchanges, or for smart contracts that are less concerned with getting the best price marked to the millisecond. In fact I pointed out the stale order issue to people who created the one of the more popular DEX protocols over lunch. They pointed out what DEXes are for. I guess nobody sent you the memo. It's OK to not know this; I didn't know. Personally I think they're bloody stupid, but you making their use out to be something sinister or poorly understood is ... well, a lot like Michael Lewis paranoid lunacy.

I realize you're not Michael Lewis, but you're taking on the title of his execrable book to sell ... something much less science fictiony and exploitative than his tall tales, but still fairly off the mark.

Re:Trading bots are running wild on cryptocurrency exchanges by phildaian on Tuesday April 16, 2019 @09:39AM (#19673456)
Yes, the people who created DEXes do know about and advertise this issue, because this is work we've been doing since early 2017 and discussing with them even earlier. In fact, I advise a lot of those projects, and many consider me the SME on DEX frontrunning. I'm also the reason 0x e.g. released a bot to explicitly do this.

To claim this is what they were designing for is just ignorance of the space though. None of them knew about or even anticipated this issue; just because it is obvious to you does not mean it was obvious to them.

Just because DEXes are only for regulatory arbitrage (your words, not mine) doesn't mean that we should stand up crappy market designs.

The majority of the paper is about the formalization of and data on PGAs and miner OO, which was also not known in the blockchain community until now. If you point me to a reference to the contrary, or that contains the same results as e.g. Sections 5-7 of our paper, I'll gladly cite this and reduce my claims of novelty. Otherwise, this is something I've been working on for years, I'm going to stick with it, and it's up to peer review in the scientific community to decide :).

I hear what you are saying, but your preconceptions about Flash Boys being a bad book seem to be severely coloring your understanding of the work!

Re:Trading bots are running wild on cryptocurrency exchanges by tptacek on Tuesday April 16, 2019 @01:40PM (#19675634)
They aren't "preconceptions". A preconception about the book would be if they hadn't read it, weren't familiar with it, and had opinions about it anyways. Flash Boys is a bad book. I'd argue that your desire to immunize the book, which is largely unrelated to your paper, from criticism is harming the case you're trying to make about distsys flaws in DEX's.

The paper is good! The surrounding context you're trying to provide about how real-money markets work, perhaps not so much.

Re:Trading bots are running wild on cryptocurrency exchanges by phildaian on Tuesday April 16, 2019 @04:15PM (#19676910)
I'm not saying he has preconceptions about the book, I'm saying his opinion of the book is creating unwarranted preconceptions about our work.

The paper doesn't comment on money markets much at all, and cites pretty unambiguous peer reviewed work which is highly substantiated when it does.

Re:Trading bots are running wild on cryptocurrency exchanges by tptacek on Tuesday April 16, 2019 @06:19PM (#19677851)
Sorry, I was confused because you referred to his "preconceptions of Flash Boys as a bad book", and also talked at some length about how industry insiders don't like the book on Twitter, and also alluded to practices described by Flash Boys that insiders want to hide upthread.
Re:Trading bots are running wild on cryptocurrency exchanges by scottlocklin on Tuesday April 16, 2019 @03:53PM (#19676731)
Virtually all existing market designs are crappy. Maybe CBOT is an exception. Michael Lewis' book is a big giant marketing document for a shitty market design, FWIIW: not a preconception; I read and panned his ridiculous book.

I didn't think you made the case in your sections 5-7. 7 in particular looked like speculation to the point of paranoia (which, admittedly, is justifiable in blockchain, but for the kinds of dollar amounts you're quoting -rather unlikely). I'll read it again to make sure.

Re:Trading bots are running wild on cryptocurrency exchanges by phildaian on Tuesday April 16, 2019 @04:16PM (#19676916)
Well, we show (among many other results) that today's system is potentially unstable assuming miners are rational and profit maximizing. I don't think that's paranoia in a system that gives you 0 security guarantees unless you assume miners are rational and profit maximizing.

> Virtually all existing market designs are crappy. Maybe CBOT is an exception. Michael Lewis' book is a big giant marketing document for a shitty market design, FWIIW: not a preconception; I read and panned his ridiculous book.

I'm not saying anything promoted by Lewis is a good market design, nor that existing continuous limit order books are. Merely that from the PoV of the efficient markets hypothesis, blockchain markets are worse (and have the potential to be better, e.g. by exploring batched designs).

Re:Trading bots are running wild on cryptocurrency exchanges by scottlocklin on Tuesday April 16, 2019 @10:55PM (#19679363)
Yes, blockchain markets are worse in many ways; that's obvious. But I don't think your speculation about miners has any merit.

Your statement in section 7 seems to not understand how blockchains work. I could be wrong; it don't pass the sniff test. Since I actually have a day job, I may not have time to look into this in enough detail to thoroughly pull it apart, but hey, maybe they'll catch it in "peer review."

Re:Trading bots are running wild on cryptocurrency exchanges by phildaian on Wednesday April 17, 2019 @05:29PM (#19686647)
If you find something wrong with it, consider this an open offer for me to revise or withdraw the paper as a result.

That being said, I've been studying blockchains for years, and this paper has been reviewed by some very high caliber designers of blockchain protocols in both academia and industry. It also uses very simple models as in http://randomwalker.info/publications/mining_CCS.pdf which in my view are extremely unlikely to be incorrect (of course, anything is possible).

Re:Trading bots are running wild on cryptocurrency exchanges by thedufer on Monday April 15, 2019 @10:52PM (#19670754)
The reason you're getting pushback on using the term front-running to describe this is that it is a blatant misuse of words for the purpose of painting a behavior as bad without needing to say why. You've taken a legal term that describes a crime and extended its definition to something that is not a crime to make that non-crime look bad. There might be some good ideas in there, but relying on this deceptive rhetorical technique implies otherwise. If you have something worthwhile to say, why the deception?
Re:Trading bots are running wild on cryptocurrency exchanges by rmah on Monday April 15, 2019 @07:46PM (#19669875)
The "scientific community" does not get to define terms used by the financial industry and financial regulators.
Re:Trading bots are running wild on cryptocurrency exchanges by yeahwhatever10 on Monday April 15, 2019 @06:45PM (#19669573)
Except trading doesn't have a "scientific community"; they have legal definitions not scientific ones.