The important point here is that Damore triggered a massive threat response in the colleagues he characterises as being below "the bar". He explicitly talks about that bar being lowered, which by implication undermines a proportion of employees at Google (and people that are inclined to defend that group).

Furthermore he attacked 'diversity' hires as a whole, but only presented evidence on male/female differences not racial ones... so there's significant precedent for him making points that aren't backed up directly. I don't think he should get the benefit of the doubt there with regards to subtlety of meaning.