Re:Large-scale cluster management at Google with Borg
by nrr on Thursday May 07, 2015 @01:14PM
(#9506647)
From the MapReduce paper: "The MapReduce master takes the location information of the input files into account and attempts to schedule a map task on a machine that contains a replica of the corresponding input data."
One of two possible designs immediately come to mind: (1) The MapReduce master interacts with Borg directly on behalf of the programmer and places a requirement on the task to run on or near any node that has a replica of the data required or (2) for each Borglet, there is an accompanying MapReduce agent running to accept RPCs from the MapReduce master.
One of two possible designs immediately come to mind: (1) The MapReduce master interacts with Borg directly on behalf of the programmer and places a requirement on the task to run on or near any node that has a replica of the data required or (2) for each Borglet, there is an accompanying MapReduce agent running to accept RPCs from the MapReduce master.