The usual side channel is timing. Some numeric operations take longer than others depending on the key, and an adversary can measure these to narrow down possible keys: https://en.wikipedia.org/wiki/Timing_attack
Javascript engines have complicated numeric type conversion rules, together with single-entry type caches that seem very likely to leak.
While I agree that absence of evidence is not evidence of absence I have to take you up on that second statement. I'd say you deserve an encryption that's carefully constructed to provide appropriate protection against a threat model matching the use case.
A case in point would be MD5. MD5 is vulnerable to collisions, does that mean that we should use MD5 for passwords? Of course not but because of moore's law and scrypt, not because of collisions. Should we use MD5 for hashing content? For anything forensic, no. For situations where collisions aren't important, perhaps yes.
Many software crypto systems are susceptible to direct leakage through RAM, that doesn't mean that we shouldn't use them.
Why assume the risk?
I see where you're coming from with it but to take your point I can pull keys out of a memory dump, who cares which process it comes from? In this case does it mean we should all wait for a perfect OS that scrubs memory on everything properly and encrypts swap?
That's not to say you're wrong, I think you have some valid points but in every other domain it appears there's a good enough level and when I at least encounter UK government crypto we're told it's the same. The thing about the cryptocat thing is that there are questions about transparency that are valid (and I've seen your conversation on twitter and agree with some of your points), but I'm trying to avoid falling into that situation.
What specifically are the side channels you see that Javascript exposes?