That said, there are existing solutions for this, they just suck, e.g.:
http://www.seagate.com/www/en-us/products/laptops/laptop-hdd (only 4GB Flash, WTF?)
http://www.bit-tech.net/news/hardware/2010/02/03/silverstone... (this is just a dumb partial RAID)
http://en.wikipedia.org/wiki/ReadyBoost (designed to work with USB sticks of all things, and isn't terribly smart about what to cache)
Plus, that sort of file-based system doesn't cache file system structure/metadata, which is a major source of disk head movement.
If you only store your OS and programs on the boot drive, space isn't a big concern and you gain a lot of flexibility. It's less hassle to repave the OS, migrating your data to new machines is simple, and it's easier to upgrade the boot drive as faster ones are available. That's especially nice with SSDs because you no longer have to care very much about the write limit. If my SSD boot drive wears out, I have 0 data recovery/restoration to worry about.
I'm thinking of it myself, but I also thought it out further and concluded that it could be completely automated. In my case, if one just kept all files larger than 100MB on the HDD, that would be enough. Better still, I could specify certain directories to be kept on the HDD. I could do all of the above with symlinks, but setting that up once and getting the desired result automatically could be even better.