Why did you say you could do fast failover to LA in 2007? [1] It was the case then that every customer had their own SQL database too.
I'm not trying to be snarky, your team is obviously putting forth some heroic effort. I'm just an affected customer that is curious why the question seems to have two mutually exclusive answers.
[1] http://webcache.googleusercontent.com/search?q=cache:lHEK939...
In all fairness, Joel recognized in the article that the initial decision was based on their previous experience with Microsoft -- and AWS was still incipient in 2007 -- so it made sense at the time. I doubt this is still the case. Even less if you consider the costs of Sandy - duplicated servers, migration, hours lost, unhappy clients, etc.
And maybe this sounds snarky, but AWS is your "making it better" solution? Seriously?
NoVA didn't get hit that hard by Sandy, unlike NY/NJ. So data centers around here didn't tend to have problems, the few that I have equipment in didn't even loose utility power.
Separately, Stack Exchange, which is in the same data center, is running off of its hot backup in Oregon.
We would have liked to have completely redundant data centers for FogBugz on Demand and Kiln on Demand to avoid even a few hours of downtime, but because those services rely heavily on giving every customer their own SQL database, there is almost no reasonable way to get fast failover to a different data center. We can do it with Stack Exchange because there are only a couple of hundred databases. We've been building a SAN solution which will make it possible to hot swap out to another datacenter, one day, but that project is not complete.