Especially when it indeed is "crappy software" that will take major effort to rewrite/refactor.
Hardware is cheaper than developers, and the former doesn't fail to deliver half as often as the latter...
Also, you need to compare the cost of a single developer who fixes innodb to the cost of hardware incurred by _all_ users of innodb who would benefit from the solution.
Trying to fit high I/O workloads on spinning disks is a lot like spending your efforts to make your program run on 64K RAM. An interesting challenge, that would surely exercise your coding skills, but would not deliver the optimal performance.
It is hard to see the benefit of ignoring SSD to get high I/O performance vs. redesigning the software around I/O bottlenecks.
Redesigning software to work on bad hardware is definitely a challenging project, but not always the best use of resources.
SSD might be a solution, but we don't have any proof either way, and we also don't understand why is this problem happening.
SSDs really help this process happen faster because they have way better random write performance than hard disks.
=============================
What to do?
I wish I could say you should use Percona Server with XtraDB.
If we were using SSD as storage, then I would recommend it. Vanilla MySQL performs equally bad on SSD and HDD, while for SSD in Percona Server we have “innodb_adaptive_flushing_method = keep_average”.
Unfortunately on spinning disks (as in this case), Percona Server may not show significant improvement. I am going to followup with results on Percona Server.
=============================
SSD is the solution because unlike rotational disks it allows for lots of random accesses. While they are too expensive to use as "archiving" storage, SSD excels as a layer between traditional disks and RAM. Put your high I/O workload on SSD and keep your historic data on spinning disks.
If you want to see some SSD benchmarks, here is your resource: http://www.ssdperformanceblog.com/
As far as cloud storage, yes, Amazon EC2 offers 68GB RAM, however if you need hardware of that size it is not hard to justify setting up your own servers. It would be cheaper too and you'll be able to fully optimize your storage.
My macbook pro I upgraded with a gaming grade SSD gets 100-200 mb/s random write.
Just upgrading to SSDs is more viable than hacking up a database with a huge community behind it that you didn't originally create yourself.
If we were using SSD as storage, then I would recommend it.
Vanilla MySQL performs equally bad on SSD and HDD,
while for SSD in Percona Server we have
innodb_adaptive_flushing_method = keep_average.
Doesn't seem like a solution at all, if we believe the author.Given that Percona is fully backwards compatible with MySQL, while adding this and many other tuning options, I just cannot see a reason to use vanilla MySQL ever.
Solid State Drives is the easiest solution to most problems requiring high I/O throughput and data persistence.
Too bad most cloud hosting vendors are way behind on offering anything like that.
EDIT: Not sure why this comment got down voted, but perhaps whoever did it might comment on why using SSD is not a solution.