Unfortunately, from a security perspective, moving those things to software doesn't make the vulnerability go away. You can have the same vulnerabilities in your software implementation.
From a performance perspective, we have no general solution to parallelizing a serial program. GPUs used to be VLIW. AMD switched from VLIW-5 to VLIW-4 because the average width was only ~3.5 (Nvidia had switched from VLIW to SIMD long before this). Today, Nvidia and AMD both use a MIMD threaded approach to execute on SIMD units.
Later-generation Itanium chips wound up including branch predictors and speculative execution. From what I understand, under the hood, they were normal RISC-style processors (like all the x86 micro-arch are today). Just ignore the VLIW and run one set at a time serially with the ILP hardware optimizing as it goes.
In today's programs, the programmer specifies data-level parallelism where possible (and if necessary, re-adjusts the code so the compiler heuristics recognize it as optimizable). The compiler then tries its best to detect the parallel data and use SIMD and organize instructions so that the parallelizable ones are closer together (so they fit in the CPU reorder buffer). When they hit the CPU, it examines the code as it runs to optimize speculative execution and uses the reorder buffer to make efficient use of its computation units (load, store, ALU, FPU, SIMD, etc). You move from explicit to implicit-ish (you know that putting similar instructions will optimize in all modern processors), but don't have the drawbacks of noop code bloat or having to compile different code when someone changes from VLIW-2 to VLIW-3 code.
Can you elaborate on how the halting problem relates to the IA-64 architecture?
In the context of IA-64, that means that generally, the compiler will not be able to determine when it's save to use parallelism. We can only program it to use parallelism in a bunch of special cases for which we think it will be safe.
Given that simias originally asked for a simpler platform that we can trust in more, this seems to be a pretty strong argument that IA-64 is not that architecture.
"Leaves branch prediction and speculative execution up to the software" means that in practice you need to either use the Intel compilers or hand-optimise your software to get the benefits, while simply recompiling your legacy software with GCC or LLVM ends up being disappointingly slow.
Intel could have solved this problem by contributing to GCC and LLVM instead of keeping their optimizations proprietary. Then they would also be more easily auditable as well.
There were no magic secret optimizations to release. It just straight up did not work. They had to add back dynamic branch prediction, and even then the load store latency was such trash that they had to put ginormous L3 caches on it to get even close to reasonable performance.
Yeah, compilers today are no better. We found the limits of statically scheduled parallelism pretty fast. On code that uses static scheduling, a modern OoO processor can easily duplicate what IA64 was capable of (and a pipelined loop using AVX will utterly smoke it), while being far better at all the stuff IA64 failed at.
I remember going to a corporate presentation by Intel about the revolutionary CPU architecture. They had to turn the machine off because the cooling fan was making so much noise we couldn't hear the speaker.
And this at precisely the wrong moment in history, as mobile devices, with their stringent power consumption constraints, were becoming popular.
In particular, you end up losing the ability to avoid stalling on cache misses, and the memory wall becomes an increasingly gigantic problem even as Moore's Law progresses.
We don't need to give up on performance, even with low-performance RISC-V chips. What we need is a new architecture which takes advantage of low-speed multicore processors. If it is possible to produce thousand-core RISC-V chips [1] then any core could be assigned a single thread - which implies that there is no need for a complex multitasking OS.
[1] Towards Thousand-Core RISC-V Shared Memory Systems (PDF)
https://riscv.org/wp-content/uploads/2016/11/Wed1000-Thousan...
What would have been the right ones?
PS3 with Cell would have been a failure had Sony not eventually caved in, and released a PS3 SDK that made most of the work Sony was initially expecting devs to do, regarding low level programming.
That work became PhyreEngine.
IA-64 (https://en.wikipedia.org/wiki/IA-64#Architecture) already exists, with its explicitly parallel instruction set, which leaves branch prediction and speculative execution up to the software. So you can still get many of the performance benefits, but it's under control of the software, giving a lot more flexibility for being able to mitigate or eliminate these kinds of issues.
I think that it was probably introduced ahead of its time, and targeted at the wrong markets, but it's kind of sad that with the Spectre vulnerabilities, we don't have any way of comprehensively addressing it in software without jumping through a lot of hoops and applying microcode updates.