I believe "Spectre" is an ever-growing series of ways to read the memory contents of another process by abusing a CPU's ability to do speculative execution. It's unlikely that a CPU will be available in the near-term that's immune to it, so the only sure fix is to use air-gapping. So in principle, your suggestion is vulnerable. Alternatively, the problem may be academic. There is a still-growing class of Spectre attacks, so it's hard to know how flawed your comment was.

The only similar bug I'm aware of was Meltdown, an Intel only bug that was immediately patched with a microcode update. But Meltdown was a different bug to Spectre. Spectre is a class of attacks that's hard to solve by design, Meltdown was a specific bug that was easy to solve.