Re:One trillion edges: graph processing at Facebook scale [pdf]
by Smerity on Thursday August 13, 2015 @12:02AM
(#10052195)
Other posters have replied giving a back of the envelope as to feasibility, so from the other end:
The PageRank implementation on Dato's GraphLab Create when run on the Web Data Commons Hyperlink Graph (128 billion edges) does 3 billion edges a second on 16 nodes, which is 187 million edges per second per machine.
Given that communication overhead quickly becomes an issue for most of these systems and their graph is dealing with more edges, 28 million edges per second per machine seems quite reasonable.
The PageRank implementation on Dato's GraphLab Create when run on the Web Data Commons Hyperlink Graph (128 billion edges) does 3 billion edges a second on 16 nodes, which is 187 million edges per second per machine.
Given that communication overhead quickly becomes an issue for most of these systems and their graph is dealing with more edges, 28 million edges per second per machine seems quite reasonable.
[1]: https://twitter.com/CommonCrawl/status/623615774909857792