Re:Concurrency's shysters
by swannodette on Monday July 26, 2010 @04:38PM
(#1549199)
Except those ACM microbenchmarks are exactly the kind of fine-grained problems Rich Hickey specifically stated he's not trying to solve.
For example http://dosync.posterous.com/stm-couchdb-and-pushing-5500-doc.... I added only 7 lines of code. Raw Aleph runs at about 20k req/s on an 8 core box. This code performs at about 14-18k req/s. So clearly some contention is at play. I could write this with locks, but I have better ways to spend my time. That's the real cost-to-benefit ratio.
Stuffing a bunch of things into a DB in parallel doesn't seem like a very interesting problem to solve to me. As discussed earlier, Clojure's concurrency primitives are only useful for a somewhat specific class of problems: computationally intensive tasks in a single shared memory space. The benchmarks in the ACM paper seem to me to have a lot more to do with the kinds of complex datastructures you're going to employ in those scenarios than batching writes to a key-value store.
tehse 2 posts were useful in thinking about taxonomies/how to categorize concurrent/ threaded (and data parallel) tasks
http://blog.ezyang.com/2010/07/graphs-not-grids/
http://www.bluebytesoftware.com/blog/2010/07/12/ThoughtsOnIm...
For example, I'd like to see if a Clojure solution of the benchmarks in the linked ACM paper can scale better than the solutions tested in the paper.