Whether you view that attack model as something worth considering depends entirely on context. But it's a valid view for many applications. As long as people don't use it with any expectations of security under active attack models, I'd say that's okay.
My argument would be that trying to protect against passive attackers with JS adds nothing beyond what SSL already offers.
Which is already required as a matter of course, and already compromises the payload if SSL is broken (again).
Again, I am not advocating JavaScript crypto. I'm just putting it in its place.
As pointed out elsewhere in the thread, there are few attacks that allow you to listen in on an SSL connection's content without also allowing you to modify that content - say, with a version that pastebins your keys.
Hence my argument that JS cannot provide anything SSL lacks, plus or minus some wishful thinking. Combine this with the fact that it's impossible to protect against a MITM-modified JS payload (see the "chicken-egg problem" portion), and you have a rather uphill battle here.
On the other hand, plenty of people who can modify the traffic exist. Starting with every employee at your ISP and the Amazon picker who shipped your router.
The NSA are well resourced, but I doubt they have much more than a second spare for every internet user on the planet. So speaking as a relatively uninteresting person, I actually would find that somewhat reassuring. If you're a Person Of Interest, that's obviously a very different situation, and you shouldn't use browser-based crypto.
So, yeah - if you rolled your very own library that's unique to this planet for exactly one website, congratulations! You're secure as long as there are no attackers! Doesn't really say anything useful about your security though. Or about the viability of using a fundamentally broken crypto platform to do crypto.
Most CVEs that come to mind assume a somewhat determined attacker.
It takes a reasonably determined attacker to commit to rails without permission [1] or run a ten-line perl script to crash a server [2] too.
Waving away a problem via "the bad guys would need to think for more than one second" is not exactly reassuring.
[1]: https://github.com/rails/rails/commit/b83965785db1eec019edf1...
[2]: http://www.ocert.org/advisories/ocert-2011-003.html