Upgrading from Rails 3 to 4, though, feels much easier as there haven't been so many fundamental changes in the way Rails does things. And things that have changed have usually been moved to gems that you can include to keep your app working as it was before.
Fundamentally though ... Rails has an attitude that makes it pretty difficult to adopt at big organizations precisely for the reasons that you've mentioned. I'm yet to encounter a Rails version upgrade that wasn't a massive task , and I can imagine having to deal with that as a startup whose life's blood depended on it. (airbnb's experience on this is here http://nerds.airbnb.com/upgrading-airbnb-from-rails-23-to-ra..., and lots of companies don't have the resources or talent that they do). And supporting older versions in the Rails community does get short shrift though it is getting A LOT better these days.
However, I have to take issue with your assertion that
"I am also strongly convinced that Rails is never the right solution"
Which I feel comes from a place of frustration. I think the rewards of using Rails ... developer happiness, productivity and ease of getting from idea to software, make it worth it to put up with some of the warts that it has. Things will only get better (and already are).
The best part about the whole thing is there’s potentially 10 new web frameworks that can take on Rails. Hell, Mongrel spawned or helped 5 of those. My favorite of those frameworks is Merb which is literally “Mongrel plus Erb”
Also, I'm not sure that the best test case for the ease of rails upgrades is embarking on a major migration of an app you don't know well, written in a previous version of a framework you don't know well. I've been put off frameworks by bad apps before too so I know the feeling - is it possible that your experience is related partly to this app, not just to Rails? Many others have not experienced anything like this level of pain with Rails upgrades. I updated a terrible drupal site once - let's just say that I'll never touch drupal again, but that is not necc. justified, it runs a lot of sites without drama and can probably be used to make good apps in the right hands.
Certainly if you are familiar with the app and gems used minor upgrades are exactly as described above - upgrade, test, deploy, sometimes with no changes. That's not to say there are no problems, but I think you're extending your horrible experience unjustifiably to encompass all possible uses of Rails. I find it not bad as frameworks go, better than many others I've seen internally, particularly > v3, and plenty of other people other than Rails developers use and enjoy Rails, without any resulting disasters, why the hyperbole?
I'm personally very thankful for all of the hard work the community has done to help us upgrade.
If you have found pieces of the upgrading puzzle that are missing, then please consider contributing back. 2.3 > 3 is the hardest one. It gets a lot easier.
Worst case, your app is exceptionally hard to upgrade and I feel for your effort.
In what way was McAnally's book insufficient?
http://www.railsupgradehandbook.com/
> I've led upgrades from PHP 4 to PHP 5 which were both less painful and more quickly completed than the task of upgrading a single, rather small, site, from Rails 2.3 to Rails 3.2 has been.
PHP is a language and Rails is a framework. Compare upgrading PHP to upgrading Ruby. The latter is painless.
It seems like you should stick with PHP.
Everything in Rails is so much work because Rails is a rolling disaster; every new major version replaces enormous chunks of framework functionality with enormous new chunks of functionality, which imposes a heavy load on the developer; "convention over configuration", plus major changes to the convention du jour in every major version, equals a necessity to re-learn, at every major version change, where the new magic lurks and precisely how it works. If I'm being entirely fair, I can't say that the Rails developers have no interest in backwards compatibility, but such interest in my experience is both quite spotty and heavily concentrated on obvious and conventional behavior, which means anything non-obvious or unconventional is probably going to need heavy rework. (I have yet to run across a Rails app which did not, at least occasionally, do something non-obvious or unconventional, and necessarily so; no framework can cover every use case, but Rails in particular seems interested in covering only the most common ca. 60%.)
To make matters worse, no one in the Rails community has the slightest interest in yesterday, and even today tends to be neglected in favor of tomorrow. You talk of entire books on the subject of Rails deployment, and while the necessity for such books to exist is certainly a problem, actually having such a book means you're doing better than average; I've recently been fighting my way through an upgrade of a Rails 2.3 app to run under Rails 3.2; it's a difficult enough task to begin with, and is massively complicated by the fact that no real documentation exists on how to perform such an upgrade. What's there is mostly to be found on various Rails devs' Tumblrs, or in IRC logs or commit comments, and these are both poorly curated and evanescent; I've spent quite as much time trying to collate information on how to perform this project as I have actually editing code, and if not for the Internet Archive, a couple of critical Tumblr posts would no longer be available at all, thanks to the Rails dev who wrote them having apparently decided to "reboot" his blog.
The whole process has been enormously frustrating, not to mention slow; I started this project better than six months ago, as a novice Rails developer, and I'm just now getting to a point where I can see the light at the end of the tunnel. I am also strongly convinced that Rails is never the right solution; as long as I'm employed at this organization, it will embark upon a new Rails project only over my metaphorical smoking corpse. There's just too much to go wrong, and the underlying ideas of the Rails community are too wrong-headed to make for a reliable platform; the purpose of a framework is to free the developer from having to rearchitect the same infrastructure over and over, yet Rails not only forces developers to do so during upgrades, but the massive changes to core functionality I mentioned earlier make the rearchitecting task enormously more complicated and painful than it'd be in the absence of any framework at all -- I do not engage in hyperbole when I say that I've led upgrades from PHP 4 to PHP 5 which were both less painful and more quickly completed than the task of upgrading a single, rather small, site, from Rails 2.3 to Rails 3.2 has been.
Rails, overall, is the worst example I've ever seen of the development model which Zawinski calls "CADT" [1]; between that, and the deep-seated and dangerous problems Shaw identifies with the Rails community as a whole [2], anyone other than a Rails core developer, who involves Rails in a project of any sort save perhaps a tiny single-developer throwaway prototype, is borrowing trouble on a potentially catastrophic scale both for himself and for any unwitting organization on which he chooses to inflict the resulting disaster.
[1] http://www.jwz.org/doc/cadt.html
[2] http://harmful.cat-v.org/software/ruby/rails/is-a-ghetto