I did this in one of my projects (Ajax-Gist.com) where I was directly querying a 3rd party API (GitHub) for each request. That meant that each request was client->server roundtrip time plus server->api roundtrip time, which worked out to be around 200ms, or a total throughput of 5 requests per second.
This is obviously unacceptable and a complete waste of resources so I re-wrote the whole thing to be asynchronous. The new application, running in a single thread on the same hardware can now handle around 70 requests per second with 30 concurrent users. That's pretty damn fast for a Rails application...
Now I didn't use Node to solve my problem (though I considered it!) but the principles and the message I took away was the same. I think it's a handy tool to have in your toolkit, but as you've already realized it's simply not necessary for a lot of projects, particularly smaller ones.
What library/technique did you end up using to make your app asynchronous?
Link: http://ajax-gist.com/
In the end I built what was essentially a proof-of-concept asynchronous Rails 3 application. The libraries I used under the hood to get everything async were:
* Rack/Fiber-Pool: http://github.com/mperham/rack-fiber_pool
* EM Synchrony: http://github.com/igrigorik/em-synchrony
* Eventmachine: http://rubyeventmachine.com/
Using Rails for what is essentially a thin proxy is overkill, and I could have gotten better performance with Node or by writing a Rack application directly. But I wanted to take the opportunity to get a handle on what would be involved with doing this in Rails, as not too many people have done it before that I am aware of (and at the time no-one had done it with Rails 3).
I know just enough about GNU/Linux I/O to be dangerous, so I can start to build an inkling of understanding based on this information: "watching" a file descriptor is probably how we register event listeners, and "edge vs. level" is most likely a decision about when handlers receive their data. But that's about as far as I can get before I have to start descending into the terms I don't completely understand. When I find something that explains those, it will probably rely on a bunch of other terms that I must descend into, and so forth. By the time I finally recurse all the way back up to my original quest for knowledge ("What does event-driven mean?"), I will be so inundated with new information that nothing sticks!
Thanks for the suggestion about Twisted, EM and Node.js... I'll have to make some time to play with them soon.
Even just playing with writing some meaningful client-side JavaScript will be enough to give you more of a 'feel' for event-driven programming...
It's not going to give you a huge amount of knowledge straight out but it will definitely give you enough to get you started if you decide to go further.
I put "synchronous, I/O-bound" in scare quotes because I don't really grok what it means. Sure, I can talk the talk and mumble a few buzzwords under my breath, but when it really comes down to it, I have only a cursory understanding of what "event driven," "asynchronous" or, Hell, even "single threaded" means. I've read the Wikipedia pages and made it to "Hello world" with a few of these concepts, but I feel like I'm really missing a huge chunk of very important knowledge & experience.
What kind of project can I work on that would go down this road? Are there any good books, physical or electronic? Or am I doomed to be disconnected from this universe of experience until one of my crappy web projects gets Digged and I learn through trial by fire? I just know there's more to this than "asynchronous method calls will not block method calls that come after them."
EDIT: I noticed that someone downvoted this--I'd be extremely interested to know why. Am I missing something incredibly obvious?