News for hackers. Stuff that matters.
Re:A Trillion Edge Graph on a Single Commodity Node by kajecounterhack on Friday April 28, 2017 @03:24PM (#14222310)
"Efficient processing of large graphs is challenging. Graph algorithms often exhibit poor locality of memory access, very little work per vertex, and a changing degree of parallelism over the course of execution [31, 39]. Distribution over many machines exacerbates the locality issue, and increases the probability that a machine will fail during computation."

https://kowshik.github.io/JPregel/pregel_paper.pdf

^ (Pregel is Google's batch graph processing framework)

FWIW most graph computations can fit on a single powerful machine these days. There really aren't that many that require distributed processing.

Re:A Trillion Edge Graph on a Single Commodity Node by anonetal on Friday April 28, 2017 @05:54PM (#14223412)
The parallelization problem doesn't really go away though, since you still need to figure out how to do that to exploit all the cores effectively.
Re:A Trillion Edge Graph on a Single Commodity Node by rspeer on Friday April 28, 2017 @08:07PM (#14224176)
You don't need to. You're never forced to solve really thorny problems to make your code run faster by a constant.