It's not a no-op, it has to check types and maybe convert representations, much like a C/C++ FFI (but with more freedom for the implementor, who's not committed to the C ABI, so there's potential to go faster than a C FFI).

But I agree it needs attention, and apparently so do the Firefox devs. In http://wry.me/hacking/Turing-Drawings/ the asm.js version ran much slower than the regular JS version, until I adapted it to make fewer calls. The slowness also makes some other uses I've thought of impractical.

(The figure I gave was not carefully measured; just enough to say "yep, that'd account for why this is amazingly slow".)