I mean I'm as against software patents as the next guy but this isn't the most trivial one I've seen. Glad they licensed it to ASF.

Adding all kinds of bells and whistles to it does not make it 'novel'.

Redundancy, failover, node locality and rack locality have all been done many times over in the past, maybe not in that context but that is just a combining of concepts.

To be awarded a patent you should at least do something original.

Putting that aside, what Google did with MapReduce was original. There are a number of challenges that each of the individual concepts never had to overcome, which required new, unique concepts to be introduced to make the system work.

Only if the mechanism is novel. And that's the problem here, the mechanism is identical, just larger.

Running map/reduce on an array or on a petabyte data set does not change the fundamental method at all. There are just countless implementation details to be taken care of, but any competent programmer could extend the original concept of map/reduce to that level.

It very much changes the fundamental method. It takes something that was once impossible given all existing techniques and technologies, and makes it possible. The new techniques and technologies are at the core of the patent.

any competent programmer could extend the original concept of map/reduce to that level.

Patently false. At the very least, you need a decent sized team of very competent programmers, DBAs, and system administrators to architect and build the system, and another programming test team to ensure that it is reliable. And then you'd only have a working prototype. This isn't nearly as trivial as you make it out to be. Hadoop is evidence of that, it's going on 4 years and four thousand changesets, with changes by over 50 people from Apache, Yahoo! and the general public. Its source directory (for only the common project, not including the HDFS) clocks in at 328 KLOC.

MapReduce wasn't some little side project hack-job.

But then again, you could probably code up Stack Overflow in a weekend, couldn't you?

Thanks for playing.

>Only if the mechanism is novel.

Actually, no. It's perfectly reasonable to patent using old mechanisms to do new things or to do old things differently.

For example, appropriate to patent using a lever to do something that no one had thought to do with a lever before, despite the fact that levers are one of the oldest mechanisms.

Note that new mechanisms don't come along very often. Much of what we get are new uses or new applications.

I already said I'm against software patents -- but this is one of the worse cases to back up the viewpoint, they actually invent something here.

The first time someone did map/reduce, the first time someone did quicksort, that was invention. Doing it larger or through some other variation does not make it an invention.

Anyway, you seem to be set on your position that they invented something, I really think they didn't. Map/Reduce to me is actually a poster child for a very bad software patent.

I'll admit that it's a fuzzy line and you can make a case for this particular thing to be on either side of it -- but having components that are previously established isn't enough IMO to definitively say "nothing was invented".

cat somefile.txt | grep sometext | cut -d ' ' -f 1 | sort | uniq -c | wc

Is technically in violation of the map reduce patent, it is that broad.

Go read the thing, seriously it is very interesting reading.

The meat is this section:

"A large-scale data processing system and method includes one or more application-independent map modules configured to read input data and to apply at least one application-specific map operation to the input data to produce intermediate data values, wherein the map operation is automatically parallelized across multiple processors in the parallel processing environment. A plurality of intermediate data structures are used to store the intermediate data values. One or more application-independent reduce modules are configured to retrieve the intermediate data values and to apply at least one application-specific reduce operation to the intermediate data values to provide output data."

I'm just saying that what they've implemented is qualitatively different and an invention compared to your clever shellscripting, even though you can call them "sort of the same thing" from a sufficiently philosophical point of view.