I think most of the opportunity for "invisible" performance improvement today is below the ISA -- new branch predictors, new prefetchers, smarter cache hierarchy eviction/replacement heuristics. These are problems that are fundamental to the code: predicting dynamic behavior and having the right data nearby. (At least, this is what I saw again and again while staring at pipeline traces while working on a commercial microarchitecture.) A new ISA won't fix that.

Or maybe said in a different way, even if we recompile from C to a new ISA, we still have to execute the code that (by virtue of its algorithm) has unpredictable branches and lots of cache misses.

I do think there's room for innovation at a higher semantic level, e.g. maybe a really fast thread message-passing/synchronization mechanism with a lot of hardware thread contexts. Or lots of other ideas, e.g. "helper threads" (user-provided speculative prefetch code) or "informing memory operations" (load-and-branch-on-cache-miss). (All of these come out of comparch research in the 90's and 00's.) But all of those would require programmer cooperation, since they introduce new semantics.