Re:Ask HN: To everybody who uses MapReduce: what problems do...
by lclarkmichalek on Sunday November 10, 2013 @01:07PM
(#6707631)
For our highly unstructured, untyped, non relational artefacts? It'd be fairly impossible to use it as a datastore in this particular case, and regardless, it would provide little to no speed increase over our current application, as the limiting factor is the CPU cost of the map function.
[ Parent
| on HN ]
Re:Ask HN: To everybody who uses MapReduce: what problems do...
by alecco on Sunday November 10, 2013 @03:27PM
(#6708278)
Maybe you should consider transforming the data to structure it a bit. Have a unique identifier per object and arrays for each seen characteristic with (value, id), and it's reverse index. Then decompose the processing of each object to sub-problems matching n-way the characteristics. It doesn't have to be in SQL, though. I'd try a columnar RDBMS. YMMV