Your last point is basically what Spectre v1 mitigations are all about, at least if you throw in the word "speculation" somehow. The rule is: don't speculate past branches that depend on untrusted data (though there are certain additional considerations about what the speculated code would actually do).
It's just that there are a lot of branches that don't depend on untrusted data. Speculatively executing past them is perfectly fine and extremely valuable for performance. That's why nobody wants to get rid of speculative execution.
Why not?
> but we will effectively need to get rid of the idea that you can run untrusted code in the same process as data you want to keep secure.
If we accept that, then we also need to get rid of the idea that we can branch differently in trusted code based on details of "untrusted data".