SJCL is kind of a pain in the ass though, to be honest. It was built before Uint8Array was prevalent, meaning all your crypto data has to be encoded as a string, and you have to be extra cautious of UTF8 data (you have to decode/encode your data as ASCII strings). Someone please correct me if this is no longer the case.
Recently I've been playing with the emscripten port of libsodium (https://github.com/jedisct1/libsodium.js), which seems to be working quite well. I have yet to benchmark, but the nice thing is that eventually I can replace parts of the app with Rust (WIP) and use the exact same library for crypto as used in the js app.
So, I'd agree in general that javascript is a shitty language for crypto, or at least it was before Uint8Array et al. I'm looking forward to seeing what happens with Wasm...being able to replace a JS app with low-level code compiled from Rust or something is definitely a nice idea. My ultimate goal was to provide a comm layer between JS <--> Rust and embed the Rust portion of the app as a library (.dll/.so/etc) but perhaps it just makes sense to compile everything to Wasm and embed it that way.