But even supposing you pull all that off - spectre needs not just some entropy leakage, but also a way to put that entropy in context. Knowing a few random bits in some other processes memory space won't do you much good; you'll need to know something about how that process is working to interpret those bits and target the slow leakage, which is again pretty situational.
Finally, you need some way to check for indirect behavior changes, and typically that means timers. Those aren't directly available in WASM, and aren't as easy to replicate using other means as in a normal process; e.g. wasm doesn't even have fully capable shared-memory threads for exactly this reason (there's a proposal, and chrome feature flag, so you can play with it, but that's it); and other tricks to get reliable timers like via SharedArrayBuffer have been tweaked to avoid attack surface. But note: that's the case for WASM, but not for a classic OS program, even one running in a VM.
I'm not sure how much of a threat spectre is in practice, but clearly WASM is aware of it and avoiding features that would increase exposure, even very valuable stuff like threads.
Innocent and naive as I am, I feel like this shouldn't be true in a post-spectre world? Surely intra-process isolation is recognized as, at minimum, a difficult problem to solve today? The attack surface of your host, given arbitrary compute powers (which wasm has), is quite significant.
It's not to say that intra-process isolation isn't a worthwhile goal, I'd rather see new systems designed with the idea if they can be. I'm just saying I don't know that "unsandboxed but has lots of mitigations" vs "sandboxed but trivially exploitable" is so clearcut.