Re:Official Google Blog: Sorting 1PB with MapReduce
by jwilliams on Saturday November 22, 2008 @10:41PM
(#373464)
This is a really good point.
I was assuming that the sort was actually a partial sort - returning the top "x" (admittedly - even then isn't linear). However, I was also assuming (n << x), which would almost certainly be the case with a large data set. It's a fairly common approach, but not sure why I automatically assumed it.
If the sort is complete, then yeah, that changes things.
I was assuming that the sort was actually a partial sort - returning the top "x" (admittedly - even then isn't linear). However, I was also assuming (n << x), which would almost certainly be the case with a large data set. It's a fairly common approach, but not sure why I automatically assumed it.
If the sort is complete, then yeah, that changes things.