Respectfully, I think you might be limiting the attack space here in a fashion that attackers are not restricted to doing. For example, there appear to me to be opportunities for side channel attacks which do not require the bad guys to totally break your sandbox model for them to be able to guess at what another Chrome process is doing. Given that they'll have a local vantage point to do the timing measurements, and consequently very high precision, it's at least possible that they can guess things about the operation of the SSH process. If the guess the right things, they don't need to cause the SSH process to execute any code at all. (If memory serves me, that's previously been used to time OpenSSL's encryption/decryption itself, but there's probably other vectors if that isn't an option these days, like timing I/O operations with sufficient precision to guess what they were.)
There's an interesting question on whether moving the SSH into a Chrome process makes it more vulnerable vis-a-vis other Chrome processes to timing attacks than it would be just by being on the same box with a Chrome process. I would not trust myself to say "That is definitely more of a risk" with the level of research I've done, but I think it is highly unlikely that it gets less risky.
The bugs Mark found in NaCl several years ago don't alter the threat model for a standalone SSH client either. And for context, Mark's work was done while NaCl was in alpha development, years before it shipped in Chrome; were performed as part of a Google sponsored contest and later a Google paid audit; and applied to the NaCl validator sandbox, not the outer process isolation sandbox. So, even the more nebulous threat you're implying is grossly misstated.