Did you use Rails 3 and Ruby 1.9.1? One of the benefits of upgrading to the latest version of Rails and Ruby is supposedly performance. As many people haven't yet converted to the new version, I wonder how much of this discussion is a byproduct of running the older software. Thoughts?

I didn't go with Rails 3, I like to keep at least a minor version behind so I don't have to deal with framework bugs in production :-p

In my case the performance issues really have nothing to do with Rails and everything to do with blocking I/O. Rails 3 and Ruby 1.9 may be faster, but that doesn't help you at all if you do something like this:

  class ExampleController < ActionController::Base
    def index
      sleep(1) # contrived example, imagine a slow API call here
      render :text => "Completed!"
    end
  end

Because Rails is setup to be run in a blocking fashion my example will only be able to serve 1 request per second. The fix here is non-blocking languages (like Node.js) or programming in an event-driven fashion (by using EventMachine or the like). For example (in pseudocode):

  when( request.received ) do
    register slow_api_request, :callback => :answer
  end
  
  def answer do
    response.send( api_result )
  end

Ideally the framework should do this by itself. Standard libraries that are blocking, like Net:Http, should continue to appear to the developer like they are blocking, but in reality would pass back control to other instructions while they're just sitting around waiting for the remote server to respond.

This is totally possible, and getting more feasible by the day. Thin (the application server) and Rack both allow applications to run asynchronously, Neverblock provides a cleaner way for providing pools of Ruby Fibers to an application to allow implicit scheduling, in time Rails will hopefully build upon this. I've spent several hours this morning doing research into how I can fix this problem for my application, some of the stuff I've dug-up is here:

  http://github.com/igrigorik/em-http-request
  http://github.com/rkh/async-rack
  http://www.espace.com.eg/neverblock/
  http://macournoyer.com/blog/2009/06/04/pusher-and-async-with-thin/
  

It looks like the solution in my case is going to be something like creating a Rails Metal endpoint for Rack, that completes the Net:Http requests using em-http-request. I'm not sure that this is actually possible yet, but it'll be something close to that.

He's waiting for external network I/O. It's about concurrency, not performance. Even if he wrote the app in C it's not gonna help him unless he increases concurrency quite a bit.

Exactly...

Are you able to cache the results of the external HTTP calls (assuming that's what you mean by 'network responses')? I did that for one of my Rails apps and it's handling tons of requests fine.

The issue is that there is a lot of possible external calls that could be made. Therefore huge traffic for one particular item is not a problem, but moderate traffic for lots of different items is a huge problem.

I am able to cache the results (and I already am) but I don't want to keep the cache too long (it's setup to expire after 30 minutes at the moment and that's as high as I would like to go). I'm running on Heroku (which uses Varnish for HTTP cacheing) so once the cache is populated I can handle tons of traffic.

Ideally I would like to combine non-blocking HTTP requests with HTTP cacheing. I think the final goal should be the capacity to handle around 50 uncached requests per second.

If you're curious, the application is: http://ajax-gist.com/ And the source code is here: http://github.com/aarongough/ajax-gist

Maybe you can use async Rack or Cramp? Probably neither of them are supported on Heroku though.

That's pretty much what I had in mind. I guess I'll find out soon enough whether they run on Heroku! :-p