...but not just any RTG, a MMRTG: http://en.wikipedia.org/wiki/Multi-Mission_Radioisotope_Ther...
It's able to supply a steady 125 W of power for 14 years, before dropping to 100W. The technology has been used reliably on multiple projects. Based on the longevity of the previous rovers that have safely made it to Mars it's worth considering that barring mechanical malfunctions and hardware failures (it has two main computers) the rover could turn power off to non critical components after 14 years and continue to run on 100W. It might not be far fetched to think that this rover could still be running if/when humans arrive in the 2030s.
That's far cooler than what you would get running on the latest and greatest for convenience.
That constant source is charging a battery. As long as the battery is still working it would just take longer to charge the battery.
There have been efforts within NASA to design robust OS extensions -- redundancy, watchdogs, heartbeats, checkpoints, etc. -- so that fast/cheap COTS hardware could be used, at least for some functions like image processing. This has never reached critical mass and been carried to completion, however.
So yes, as you guessed, there is significant pain and heartburn that computational capabilities are so far behind. Engineers have to spend a lot of time and cleverness squeezing, say, stereo vision processing, or image compression, into the hardware available.
The bottom line is: (a) Curiosity was designed 8 years ago so the tech. on board is old (b) everything that's on board has to be radiation hardened and (c) NASA are conservative about 'new stuff' because they need it to work (e.g. the thrusters on the skycrane were derived from the Viking landers).