News for hackers. Stuff that matters.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by latchkey on Wednesday September 18, 2019 @12:00AM (#21002279)
Having gone through this learning exercise myself, it always makes me giggle to watch others do it. People first learn about arbitrage and think they can automate it to make money (tm).

In reality, the complexities involved are enormous and the biggest issue is simply that it is almost impossible to predict when the opportunity opens. Therefore, you're always dealing with what is effectively stale data. The next step is to try to trade faster, which then exposes bugs in apis and issues with dealing with connection failures.

That said, I do think that with the proper architecture and enough effort, it is definitely possible to build an arbitrage bot. Especially when you get into more complex solutions, like 3+ way. Those that have done it and are successful with it, don't sell or give away their bots because, it is arbitrage, and to make money, you have to be better at it than everyone else. Why sell a money printer when you can use it to print money?

The other realization is that the exchanges must be running arbitrage bots against themselves. I also would not doubt that there is cross exchange collusion in place as well. They help create volume and heck.. why not? No trading fees.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Pmop on Wednesday September 18, 2019 @01:43AM (#21002645)
It would be nice and time saving to have a book documenting thoughtfully these issues.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by michalu on Tuesday September 17, 2019 @01:27PM (#20997757)
>And suddenly something changed – just in one single day we no longer were able to place orders in time for few currencies. Either some one plays with scalping (which, in theory, exchanges tries to prevent), or someone were able to prepare efficient ML model to predict price movements within second intervals, or someone just can do it faster.

Something I experienced too. I would still spot profitable trade but would be late to pick them up, most of the time. With arbitrage you also have to beat the fees and the time.

Arbitrage in the past 2 years is a loser game - (or maybe not, and I am that loser). It was still a good learning experience that eventually led to other profitable ideas.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by lifty on Tuesday September 17, 2019 @05:04PM (#21000072)
HFT shops have moved in the space so you won’t stand a chance with a VPS and a Python trader.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by tastyfreeze on Tuesday September 17, 2019 @02:28PM (#20998520)
I also shared this experience. I moved from a Python prototype to Java for improved concurrency but still ended up being beat out by somebody closer or faster.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Wednesday September 18, 2019 @12:05AM (#21002291)
yeah, choice of language is not so easy in this case: c++ seems to be industry standard, but I am curious to measure how golang or scala would improve time of development and what would be their performance.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by X6S1x6Okd1st on Tuesday September 17, 2019 @07:42PM (#21001193)
> It was still a good learning experience that eventually led to other profitable ideas.

Anything you want to share?

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Tuesday September 17, 2019 @01:41PM (#20997943)
>> With arbitrage you also have to beat the fees and the time. I have to add - coins that you use for trading is also important factor, volatility may lead to situation when you were able to increase volume of some alt-coins, but next day their price severely drops
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by arbol on Tuesday September 17, 2019 @02:37PM (#20998626)
This is why you need to build a global balancer that withdraws funds into more stable coins once they reach a certain threshold. A minimum (e.g. Max trade size * 3) should be kept in each active coin for trading only.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by brianpgordon on Tuesday September 17, 2019 @11:17PM (#21002136)
My eyebrows went up when I read "It was early 2017." I'm pretty sure I remember chatting about this at work in 2013 or very early 2014 and even then my understanding was that it was too late and that automated cross-exchange arb trading had already been done. It's exceptionally difficult to squeeze any profit out of this kind of very crowded trade. There have even been Twitter bots that have come and gone which automatically identify opportunities and broadcast to the whole world how an aspiring entrepreneur can lose a lot of money trying to make fractions of a penny if they act now.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Wednesday September 18, 2019 @12:16AM (#21002334)
That's the thing that was difficult to overcome - it was common belief and a common perception that everything were settled already.

Probably in other fields situation might be the same - judging only by facts and data can bring potential opportunities.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Sytten on Tuesday September 17, 2019 @01:41PM (#20997951)
I did something similar (https://github.com/Metabot-Tech/artis), I stopped when my bot wasn't fast enough to place order. I had a lot of fun trying to optimize round trip time with various datacenters around the world.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Tuesday September 17, 2019 @02:40PM (#20998663)
Out of curiosity what was the full timing - since the moment we get data till the moment when we sent request to exchange?

>> I had a lot of fun trying to optimize round trip time with various datacenters around the world.

To be honest - I am thinking crypto exchanges are not at this state yet: many of them relying on public clouds, some of them still send non-compressed plain text json data, etc.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Sytten on Tuesday September 17, 2019 @06:38PM (#21000793)
So I was trading between a market in Ukraine (Liqui.io, now defunct) and one in Japan (Binance.com). My server was in the US in the mid-west on a cheap provider. I found it was the best to reach either side. The round-trip to each was about 100-120ms. It's quite high, but my tests showed me that going anywhere else would increase one side so the best is to have the same time on each side. I am sure there is a lot of optimization to be done to my bot, but it had a good balance of sales/buy and I would rarely lose money.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by rebuilder on Wednesday September 18, 2019 @12:11AM (#21002318)
I built a simple BTC arbitrage bot some years ago in Python. I'm basically illiterate when it comes to programming, so it was a bit of a slog. It was possible to use it profitably for a time, netting maybe between 10 and 200 USD a day, due to arbitrage opportunities between a certain small exchange that no longer exists and Bitstamp.

My bot basically only traded on two exchanges, and only triggered trades. For any real profit you had to tie down quite a bit of money and risk having them on the exchanges. But the biggest issue I faced was execution. The connection to the API for both exchanges was unreliable. Arbitrage involves executing a trade on both exchanges, and much too often, one trade would fail to execute and instead of arb the bot would effectively take a position in BTC or USD. I concluded it was beyond my abilities and willingness to learn to try to mitigate the risks that came with. Frankly I wasn't sure the problem could be worked around reasonably.

A fun exercise, for sure!

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by ilaksh on Tuesday September 17, 2019 @12:55PM (#20997383)
Interesting because they mention many of the same issues our crypto arb had to resolve.

As far as data, there are lots of ways to handle it. Initially the old team had used kdb mainly because they literally thought that was the only possible solution. I made something called `timequerylog` that uses newline-terminated JSON files separated out by date and hour.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Tuesday September 17, 2019 @01:37PM (#20997884)
>>timequerylog For storing only or for data analysis as well?

Currently our postgres have something around 1 Tb which is already not so easy to play with.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by ilaksh on Tuesday September 17, 2019 @02:26PM (#20998502)
It's on npm. It has some simple query abilities. Also there is `tql-cli`. The basic idea is pretty simple, just that you use log files and query by date and time range and key and so you avoid querying or analyzing the entire dataset at once. Note that since it uses the disk with lots of files so much, the assumption is that you are on an SSD.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by t4ko on Wednesday September 18, 2019 @04:53AM (#21003439)
I tried to do some same platform triangular arbitrage, on paper and in simulation mode my bot was earning a few cents every second. When I started testing with real money I hit a wall when I could not find any platform that placed a bid in under 500ms. My bot was 3 milliseconds away from the server, processing everything in under a single millisecond but the exchange was just too slow to do anything meaningful. In the end I did not lose anything but the fees made it not profitable enough.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by wslh on Tuesday September 17, 2019 @05:36PM (#21000329)
I complement with our detailed experiences in "What I Have Learned From My Arbitrage Experiences with Cryptoassets"[1] and in "An Efficient Algorithm to Exploit Arbitrage Opportunities in Crypto Markets" [2]. TLDR: it is very difficult to exploit arbitrage opportunities.

[1] https://blog.coinfabrik.com/what-i-have-learned-from-my-arbi... HN thread: https://news.ycombinator.com/item?id=17249163

[2] https://blog.coinfabrik.com/an-efficient-algorithm-to-exploi...

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by maps on Tuesday September 17, 2019 @05:52PM (#21000466)
This is a really spot on take on this moment in time. I also wrote a bot circa 2016/2017 [1] and the constant exchange error handling killed any profit I would eek out of the arbitrage. I was only doing this part time while a student so wasn't a huge deal to me, but losing to an exchange throwing up an error that I had never seen before and never would again, or my all time least favorite exchange cryptsy having one side of their books 'stuck' for hours was fabulous.

[1] https://github.com/mikem5/arb2

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Wednesday September 18, 2019 @12:20AM (#21002348)
That was the most funny thing - we thought if we work only with major exchanges - we will avoid any issues related with immature systems.

Thats make me severely reconsider acceptance criteria of product to be ready to face customers :D

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by space-dandy on Tuesday September 17, 2019 @05:08PM (#21000106)
if anyone wants to disclose... how much can/did you make with a system like this (towards the dawn of the crypto hype, not now)? awesome!
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Sytten on Tuesday September 17, 2019 @06:42PM (#21000823)
At peak (December 2017 to February 2018), my small python bot was making between 500 to a couple thousands dollars per day. It was even possible to do arbitrage by hand at that time since you sometimes had a 10-20% difference between the markets.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by jotakami on Tuesday September 17, 2019 @08:53PM (#21001536)
Can confirm. Early January 2018 I made $10k in a week because the BTC/JPY price was 10% higher than the BTC/USD price. It required a partner in Japan and lots of international money remittances though...
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Railsify on Tuesday September 17, 2019 @11:13PM (#21002124)
How did you handle the delay of moving coins between exchanges?
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Wednesday September 18, 2019 @12:26AM (#21002377)
If you have enough money - you can spread it among various exchanges into pre-defined coins and just rebalancing it along the usual trading.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by agumonkey on Tuesday September 17, 2019 @10:10PM (#21001882)
any good article about coin arbitrage ? right now I'm mostly curious about the operation
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by jotakami on Tuesday September 17, 2019 @08:56PM (#21001557)
Been writing a trading bot over the past 6 months and I was shaking my head vigorously at literally every single point the author brought up.

At this point the only real “arbitrage” opportunities are through derivatives (swaps and futures). However, there is still enough inefficiency in the market for individuals to squeeze out a bit of profit.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by agumonkey on Tuesday September 17, 2019 @10:11PM (#21001887)
overwhelming amount of things to manage

nothing in college prepared me to do even 10% of this

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Railsify on Tuesday September 17, 2019 @11:07PM (#21002100)
College should have taught you how to study and learn do to something like this or any other new process.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by austhrow743 on Wednesday September 18, 2019 @02:29AM (#21002826)
University certainly didn't for me. The learning how to learn course on coursera did more in that regard than any of my formal schooling.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by agumonkey on Wednesday September 18, 2019 @01:56AM (#21002701)
Even internship at a large company was at best 20% of this kind of difficulty. This entails so many little issues (technical + business) on top of the usual data storage and logic.

Maybe I fell asleep during that class I don't know.

Re:Open sourcing crypto-arbitrage bot in Python – back to th... by villager on Wednesday September 18, 2019 @12:23AM (#21002363)
+ how to overcome uncertainties, deal with unknowns and what questions to ask
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Railsify on Wednesday September 18, 2019 @10:56PM (#21012819)
Exactly, really my point should have been that you can't expect college to teach you explicit step by step knowledge for most real world problems that will require solving. The idea is that they make you solve a small set of problems and hope that you can extract a general framework for problem solving and communication. Of course there is a lot of rote memorization of reusable axioms that are taught as well.
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by jodrellblank on Wednesday September 18, 2019 @01:10PM (#21007495)
Is there any purpose to this comment apart from being a put-down?
Re:Open sourcing crypto-arbitrage bot in Python – back to th... by Railsify on Wednesday September 18, 2019 @10:12PM (#21012571)
It's not meant as a put down. I apologize if it felt like that.