Re:AresDB: Uber's GPU-Powered Open Source, Real-Time Analyti...
by alquemist on Monday September 14, 2020 @01:45AM
(#24467166)
Not a big fan of AresDB because of the glaring IO issues. That being said, pushing
'lack of join operator was never a problem' is textbook Sapir–Whorf. 'We've trained our captive users to never ask a question that requires a join, or use convoluted workarounds if they absolutely must do so, and we are proud of it' just leaves a bad taste. I wouldn't be surprised if Pinot were to add join support in the near future...
Re:AresDB: Uber's GPU-Powered Open Source, Real-Time Analyti...
by hintymad on Monday September 14, 2020 @02:54AM
(#24467539)
True that join is nice. I was just not sure if real-time join is truly necessary for Uber’s use cases, or AresDB is so much better than a query engine like Presto to justify its effort. Or more fundamentally, is GPU the solution to expensive joins?
By the way, Uber already had two real-time analytics systems before AresDB. One is the aforementioned ES-based service, and the other is Pinot, which was owned by a Pinot contributor. I was in one of those so-called alignment meetings about using AresDB. Engineers from both the ES service and Pinot were there. It was a disaster. The engineers simply asked what AresDB was supposed to solve, and presented charts over charts to show that computation or lack of join operator was never a problem (because for analytics, data streams can be pre-joined), while efficient IO was. The AresDB team simply repeated that join was important, and parallel computation was critical.
I left the company soon after, so I'm not sure how many critical use cases AresDB has been serving since then. Hopefully they do find some sweet spot to justify the cost of developing such system.