The performance hit is less because of WASM but because the Workers platform is fundamentally defined in terms of Javascript and WASM is just a thing the JS engine has as a feature, so everything has to be proxied through JS objects and code, serialized into byte arrays, handed to the WASM, and same story in reverse.

We need WASM-native interfaces to get common to get rid of JS.