Ask HN: To everybody who uses MapReduce: what problems do you solve?

Re:Ask HN: To everybody who uses MapReduce: what problems do... by Roboprog on Sunday November 10, 2013 @12:35PM (#6707444)

It sounds to me like the poster above restructured the input data to exploit locality of reference better.

http://en.wikipedia.org/wiki/Locality_of_reference

[ Parent | on HN ]

Re:Ask HN: To everybody who uses MapReduce: what problems do... by collyw on Sunday November 10, 2013 @01:01PM (#6707598)

So assuming the data is one one machine (as I asked), why would an index not solve this problem? And why does Map Reduce solve it?

[ Parent | on HN ]

Re:Ask HN: To everybody who uses MapReduce: what problems do... by virtuabhi on Sunday November 10, 2013 @02:07PM (#6707922)

Indexes do not solve the locality problem (see Non-clustered indexes). Even for in-memory databases, it is non-trivial to minimize cache misses in irregular data structures like B-trees.

Now why MapReduce "might" be a better fit for a problem where data fits into one disk. Consider a program which is embarrassingly parallel. It just reads a tuple and writes a new tuple back to disk. The parallel IO provided by map/reduce can offer a significant benefit in this simple case as well.

Also NoSQL != parallel processing.

[ Parent | on HN ]

Re:Ask HN: To everybody who uses MapReduce: what problems do... by alecco on Sunday November 10, 2013 @12:54PM (#6707563)

It's one of the issues, yes. But I wanted to be more general on purpose.

[ Parent | on HN ]