News for hackers. Stuff that matters.
Re:Building Clojure Services at Scale by arohner on Friday September 13, 2013 @01:51PM (#6381969)
One thing I've noticed with my own Clojure service at scale, is that eventually, I want to significantly reduce my use of futures entirely.

When you're getting started with Clojure, it's easy to do everything that needs to happen "later" or "in the background" as a future. Once you go to production, you realize that about half of those futures should have been jobs going into a queue somewhere, with logging and guaranteed delivery, retrying on another machine, if necessary.

I haven't gotten to the point of writing that library, but I probably will, soon.

Re:Building Clojure Services at Scale by prospero on Friday September 13, 2013 @02:16PM (#6382122)
Hey Allen, at Factual we're working on something in that vein: https://github.com/factual/skuld. As you might expect, doing it right takes something a bit more ambitious than just a library.

If you're interested in collaborating, or just have thoughts on how it could be more useful for you, let me know.

Re:Building Clojure Services at Scale by josephwilk on Friday September 13, 2013 @06:17PM (#6383542)
Yes, I shed a tear as I had to remove all the futures. Disappointment I could not get multiple thread pools and scheduling with Clojure's concurrency primitives.
Re:Building Clojure Services at Scale by adambard on Friday September 13, 2013 @04:13PM (#6382823)
I've had great success using Lamina for in-process message queues in lieu of futures, although this does nothing towards making your system distributed.
Re:Building Clojure Services at Scale by cgag on Friday September 13, 2013 @04:29PM (#6382922)
In process queues are a good abstraction, but that doesn't really protect you in case of your app crashing, which I think is the concern more than being distributed. I may be misunderstanding though.

Edit: I was already a big fan of SoundCloud, very pleased to find out they use Clojure.

Re:Building Clojure Services at Scale by prospero on Friday September 13, 2013 @05:41PM (#6383365)
I think the point was that it's hard to make something failure-tolerant without being distributed. A locally persisted queue would avoid certain failure modes, though.