If you've got heavy random IO on your dataset (90% active, OLTP here) and/or deal with any sort of significant contention or deadlocking across your disks, I would strongly encourage upgrading to SSDs. Of course, if you can fit your dataset into RAM, go that route. But if you're well past that point, might be time to consider SSDs.[1]
[1] http://www.mysqlperformanceblog.com/2010/04/08/fast-ssd-or-m...
The only way I see this happening is if they start adding servers with SSD, where all services running on those servers support SSD too. This would basically mean a separate cluster, fully based on SSD. I don't see this happening soon.
That is, for data which is touched daily, it's already less expensive to use flash than magnetic drives. The problem with Hard Drives is that in order to get the IOPs, we always end up buying a _ton_ of drives, plus the chassis to support those drives, plus the licenses to support the amount of storage represented by those drives, etc, etc, ...
Of course, if you need all you can get you should always go for just RAM, but you can get a significant improvement with a modest investment through SSDs.
The best example would be the large data set (say 100GB with a random access pattern or 100GB of working set) with enough requests that it starting to strain the RAID system you have. At this point your medium is saturated so latency is going up and up. Getting 128 GB of RAM is expensive, but swapping those drives out for a few 100GB SSDs is not hard and may alleviate the DB bottle-neck until something else becomes the bottle-neck and you need to change that anyway. Switching to RAM wouldn't have bought you anything more than the SSDs because in both cases you are getting acceptable performance up to the next bottleneck.
I'd like to think that SSDs will get us to the point that we don't have to worry about disk latency and can just toss data on there without having the database always be the bottleneck.
I believe that for most web applications the ratio is 80% reads, 20% writes, that's why no one has noticed this behavior.
Another even more concerning issue is the wear-out, SSD cells can't take more than ~10000 rewrites. While there is firmware taking care of invalidating those blocks, eventually we can expect data loss if not backed up properly.
I think SSDs are great, as long as we understand their limits... For my laptop I have one and I love it, but I would be seriously doubtful about the safety of my information in some database using purely SSDs.
This is good news for databases. Flash drives have much better random access performance than magnetic drives. With the price of these flash drives dropping, it will soon be cost-effective to replace an entire database with flash for the IOPS increase. Right now people are using flash for extra cache, but soon we'll be able to fit the whole database in flash for a reasonable $/IOPS.
It will extend the life of single-instance databases significantly, and mean everyone can wait that much longer before distributing the database.