Rails has terrible support for database replication (we've had to roll our own). Its caching architecture is laughably bad, and the implementation was/is buggy (again, we rolled our own), and there's next to no support for the kind of robust page cache expiration mechanisms that are necessary to run a large, dynamic site (Observers are only a start). Rails is a memory hog, takes too long to start, has poor translation and locale support, and simply passing a request through ActionController adds huge overhead that cannot be escaped.
And even though you don't want to hear it, Ruby is slow -- even after working around all of the above, we still had to implement a fast page caching layer just to work around the general, all-purpose slowness of Ruby.
But like I said, write me off if you want.