Re:Concurrency's shysters
by cageface on Monday July 26, 2010 @03:18PM
(#1548962)
How specifically does Clojure's STM implementation avoid the pitfalls outlined in the ACM paper? How many real-world benchmarks do we have demonstrating Clojure's concurrency scaling?
[ Parent
| on HN ]
Re:Concurrency's shysters
by swannodette on Monday July 26, 2010 @03:30PM
(#1549002)
An informative and lengthy discussion between two experts (Cliff Click of Azul Systems and Rich Hickey): http://www.azulsystems.com/blog/cliff-click/2008-05-27-cloju...
An interesting discussion but inconclusive. I agree with Rich that it would be very useful to have an established set of meaningful concurrency benchmarks.
For example, I'd like to see if a Clojure solution of the benchmarks in the linked ACM paper can scale better than the solutions tested in the paper.
Re:Concurrency's shysters
by swannodette on Monday July 26, 2010 @04:38PM
(#1549199)
Except those ACM microbenchmarks are exactly the kind of fine-grained problems Rich Hickey specifically stated he's not trying to solve.
For example http://dosync.posterous.com/stm-couchdb-and-pushing-5500-doc.... I added only 7 lines of code. Raw Aleph runs at about 20k req/s on an 8 core box. This code performs at about 14-18k req/s. So clearly some contention is at play. I could write this with locks, but I have better ways to spend my time. That's the real cost-to-benefit ratio.
Stuffing a bunch of things into a DB in parallel doesn't seem like a very interesting problem to solve to me. As discussed earlier, Clojure's concurrency primitives are only useful for a somewhat specific class of problems: computationally intensive tasks in a single shared memory space. The benchmarks in the ACM paper seem to me to have a lot more to do with the kinds of complex datastructures you're going to employ in those scenarios than batching writes to a key-value store.
tehse 2 posts were useful in thinking about taxonomies/how to categorize concurrent/ threaded (and data parallel) tasks
http://blog.ezyang.com/2010/07/graphs-not-grids/
http://www.bluebytesoftware.com/blog/2010/07/12/ThoughtsOnIm...