Articles like this just encourage pointy-haired bosses who know just enough to dangerous and annoying.

"We're shifting over to a MapReduce-based architecture, just like Google! Google's website is only an interface to a larger architecture. It all ties in to cloud computing and SOA and..."

bingo!

Vague and misleading at best. If the author does actually understand MapReduce, he's taking the brain analogy a bit too far here.

I really doubt he does understand it. That's a terrible explanation of MapReduce, and then makes it sound like it's secret Google tech that will take over the world-- parallel computing isn't new, and even MapReduce itself has an open source clone now. It's really surprising to find an article like that on HBR.

I think he must have heard the term "Artificial Neural Network" and made some hasty inference that it means something having to do with the brain.

On a side, is it just me or has HBR putting out more link bait in the past couple of weeks?

It's interesting how much noise there is around MapReduce. I'm working as CTO of a start-up with a very large dataset (terabytes) where we need to do quite complex queries across the dataset and very quickly.

Naturally we are taking a distributed approach to the problem since hardware is cheap and relatively easy to coordinate via software.

We decided to go our own way using a mixture of some databases and raw file system access with our own application coordinating both the cluster of machines and performing the queries themselves after performing extensive benchmarking of everything.

We looked closely at Hadoop and came to the conclusion that MapReduce was just the wrong architecture for our problem. Its performance was horrible.

The moral of this story is that MapReduce is interesting, but that doesn't prevent you from having to actually test algorithms and make an informed choice. Just because Google's using it doesn't mean it's a panacea.

I didn't get through the whole article, but "Google is building a new secret weapon that has more to do with the brain than search."? Google published their MapReduce paper in 2004... http://labs.google.com/papers/mapreduce.html

errrh why is this on hacker-news? does anyone here not know what mapreduce is?

i got really excited when i was about to find out abt googles secret sauce.. sigh

Yeah its the worst kept secret in the world then !

For open source, there is hadoop:

http://hadoop.apache.org/core/docs/current/mapred_tutorial.h...

Latest version of Qt also implements MapReduce patterns (only for multi-core, not across the network). There's also good old MPI, which isn't terribly sexy, but gets the job done. I found this post rather interesting when I ran across it:

http://jaliyacgl.blogspot.com/2008/07/july-2nd-report.html

Map Reduce is hardly secret and it's not like other people such as Yahoo!, Facebook and others aren't using it. Google might have a head start but that's about it.

I wonder how long that head start will last with a collaborative Open Source project backed by some other big players (that would be Hadoop).

Harvard Business Publishing? They edited my comment.

I wrote a comment, part of which you read see here: http://discussionleader.hbsp.com/stibel/2008/12/mapreduce-go...

Then a few hours later my comment has been edited -- they removed a few sentences. (I don't remember the text I wrote exactly, but I said that: 1) the author of this article should have showed his text to at least some CS student, 2) I didn't expect that Harvard Business Publishing could publish articles by such incompetent authors.)

Here we go. They could have deleted my comment, but instead they decided to edit my speech. Why just they don't rewrite everything I said?

Great, now they deleted my comment. Better than editing it.

Their secret weapon is not their assembler language, MapReduce, but their HLLs atop that, like Sawzall, http://labs.google.com/papers/sawzall.html

I stopped reading right after I read that "MapReduce let's google use the internet to think"

Oh you should keep reading, it gets even funnier.

I am interested in learning about map reduce but I have yet to find a very simple tutorial that explains what it does and demonstrate at a very simple level how it works

its not entirely obvious and you think its more complicated than it really is.

you might want to try: http://stackoverflow.com/questions/28982/simple-explanation-...

or a whole bunch of tech talks: http://vodpod.com/tag/mapreduce

the first 2 videos should give you a good overview