The instruction-level parallelism that a CPU can extract is primarily a dynamic kind of parallelism. You can, say, have a branch that's true 1000 times, then false 1000 times, then true 1000 times, then false 1000 times, etc.--as a compiler, telling the hardware to predict true or false is going to guarantee a 50% hit rate, but the stupid simple dynamic branch predictor will get 99% on that branch.
I worked on compilers too, and I think there is definitely still work to be explored here. Really, complicated OoO speculative processors are simply recovering a lot of the information the compiler already calculated! You take this code that is in no way suited for a CPU, and you do a ton of dataflow and high-level analysis on it (that you can only know from the source). You build this graph and optimized based on these facts. And then, you throw away the dataflow graph when you lower things. All this, after painfully calculating it on the assumed basis of "Yes, the CPU will like this code" -- only for the CPU to perform that whole process over again, say "Yes, I do, in fact, like this code" -- so it can execute efficiently without stalls anyway. There's clearly a mismatch here.
I mean, don't get me wrong -- this all seems like a perfectly fine and acceptable engineering tradeoff, but just disappointing from a computer science perspective, to me, at least, that this is not unified :) Of course, VLIW vs CISC etc is one of the classic debates...
Interestingly, Aaron Smith from Microsoft Research (and one of the original members of the TRIPS project) actually demonstrated and talked about their work on newer EDGE processor designs at MSR, and even demonstrated Windows running on an EDGE processor, this past week or so at ISCA2018 (complete with a working Visual Studio target, using an LLVM-based toolchain!) Their design is quite different from TRIPS it seems. I'm hoping the talks/work becomes public eventually, but that might just be a dream.
VLIW machines allow you to provide hints about instruction level parallelism without all the superscalar on-the-fly analysis of instruction level data dependencies (so the hardware can do all that rescheduling on the fly).
I think that if interlock-free software scheduled CPUs allowed us to reach 20GHz clock speeds where complex superscalar machines were stuck at 3GHz we'd all be jumping ship - but they're not
The solution might be to actually prevent some kind of security important code from being optimized in this way. Say, forcing full cache sync in-order execution for parts of code with no resource sharing between cores.
Additionally, realize that clock speeds are not a relevant performance metric. Maybe, perhaps, it's possible for a software scheduled CPU to run at 2*n GHz, but that's not interesting, if it's slower in real-world workloads than todays tech at n GHz. And for breadth-deployment in data centers we're mostly looking at throughput per Watt. CPUs with high clock speeds don't do well in that area because semiconductor physics.
https://millcomputing.com/docs/prediction/
There's a lot less mischief you can get up to when you only execute a couple of instructions beyond a mispredicted branch rather than a hundred but mischief can't be theoretically ruled out entirely.