News for hackers. Stuff that matters.

A comment on Accidentally exponential behavior in Spark. Parent comment

Re:Accidentally exponential behavior in Spark by KptMarchewa on Tuesday June 22, 2021 @06:08AM (#27589821)
Premise of the article is very true, but the comparison itself is very biased and dishonest.

Graph problems are famously hard to scale horizontally, and represent very small percent of what people use those big data systems for. Especially if you can fit the data in RAM...

Anyway, if you're able to run your workload on a single machine, then definitely do it.

[ Parent | on HN ]
Re:Accidentally exponential behavior in Spark by thundergolfer on Tuesday June 22, 2021 @08:03AM (#27590611)
I basically agree with you. Linked COST because of the premise and the upshot of the paper, which is totally valid.
[ Parent | on HN ]