I wouldn't be surprised if people eventually come around to this point of view. It will take a while, though. The current thought is that parallelism will save everything. It could take a decade before people figure out that that isn't going to happen. Maybe then there will be a renaissance of going back and rebuilding things from the ground up. But I'm rambling.
"People who are serious about software have to be serious about hardware"... that Alan Kay has a gift for putting his finger on things.
The object information is readily accessible when you compile your program with -pg, the memory map is accessible through /proc, it shouldn't be too hard to couple the one to the other and make a nice display. If you are clever about it you might even be able to fish out what memory is filled with what type of data, and colour the map accordingly.
No millions required, unix is pretty clean when it comes to stuff like this.
Some joker had written a shell replacement based on 'doom' where you could literally shoot the processes. You had to hunt them down first of course...
A graphical environment where you can see the program as it is running, with the memory map displayed in an arcade game style window.
I can imagine that such a tool would be a great eye opener.
Show hotspots in real time, move your cursor in to the memory map to get the 'hint' to display the label of the memory location at that spot.
An advanced version of it could show you the two layers caching next to the main memory and how data is flowing in and out of the caches.
I'm pretty sure that something like that is doable.