The issue is that there is a lot of possible external calls that could be made. Therefore huge traffic for one particular item is not a problem, but moderate traffic for lots of different items is a huge problem.

I am able to cache the results (and I already am) but I don't want to keep the cache too long (it's setup to expire after 30 minutes at the moment and that's as high as I would like to go). I'm running on Heroku (which uses Varnish for HTTP cacheing) so once the cache is populated I can handle tons of traffic.

Ideally I would like to combine non-blocking HTTP requests with HTTP cacheing. I think the final goal should be the capacity to handle around 50 uncached requests per second.

If you're curious, the application is: http://ajax-gist.com/ And the source code is here: http://github.com/aarongough/ajax-gist

Maybe you can use async Rack or Cramp? Probably neither of them are supported on Heroku though.

That's pretty much what I had in mind. I guess I'll find out soon enough whether they run on Heroku! :-p