It depends on what you mean by "safely". The spectre family of exploits has no general solution and requires specific mitigations. Aside from that, Wasm provides a sandbox that disallows escaping from it. That does leave scribbling on memory inside the sandbox, which is what this paper focuses on.

So for many use cases Wasm does allow what you mention, running unsafe code quickly on normal hardware, and it is being used in production for that purpose.