News for hackers. Stuff that matters.
Re:How Hunch Built a Data-Crunching Monster by jbooth on Wednesday December 08, 2010 @08:18PM (#1985542)
Yeah, but that is hard. You probably need some clustering heuristic that autolocates nodes and edges on hosts near their neighbors, a way to distribute that work, and even still, the latency penalty for the cross-node lookups will be significant and require a lot of optimization.

Unless they've thought of some brilliantly simple method that I haven't, which is certainly possible.

Re:How Hunch Built a Data-Crunching Monster by pjscott on Wednesday December 08, 2010 @08:30PM (#1985576)
They don't have a brilliantly simple method; just graph-specific heuristics. The example they gave in the paper was that, if they've got a graph corresponding to web pages and the links between them, to group together vertices representing pages from the same domain.

So, yeah, they're going to have a lot of cross-node messaging.