News for hackers. Stuff that matters.
Re:Brawny cores still beat wimpy cores, most of the time [pdf] by pjscott on Saturday July 02, 2011 @05:58PM (#2722531)
That's part of the big win of having job abstraction systems, like MapReduce: having the jobs wrapped in higher-level abstractions that make automated profiling and fancy scheduling feasible.

It doesn't even have to be anything that fancy. It could be along the lines of "run the job a hundred times on each type of machine, and measure throughput, latency, and energy usage; then try to allocate jobs to the best kind of processor for it." That shouldn't break anything.

Re:Brawny cores still beat wimpy cores, most of the time [pdf] by gruseom on Saturday July 02, 2011 @06:03PM (#2722536)
How do you tell what kind of job you have without running it a hundred times?

Or do you mean run small portions of a large job before deciding where to commit the rest?

Re:Brawny cores still beat wimpy cores, most of the time [pdf] by pjscott on Saturday July 02, 2011 @06:43PM (#2722600)
I meant each type of job; sorry for the ambiguity. So, for example, if you have a job type that is "look over a chunk of text and return term frequencies", you could test this on a few hundred chunks of text, and then use this profiling information to guide the scheduling any other time you run a job of this type.
Re:Brawny cores still beat wimpy cores, most of the time [pdf] by cma on Sunday July 03, 2011 @01:31AM (#2723156)
Google has this covered in spades: http://research.google.com/pubs/pub36575.html