Sadly I've found that post-diagnosis, at least when it comes to something that can be 'hidden' pretty well, like Aspergers, even just mentioning the diagnosis often makes things worse (especially around said assholes).