That would be very disturbing, but that's not what's written in the document. Saying that there are average differences between groups, and that therefore people should be treated as individuals, is not at all like saying that all individuals in one group are inferior to another.
I think he was wrong on some facts, and also didn't explain things in the best way.
But I think it was wrong of many people to misrepresent what he said (as "women are inferior"), and it was wrong of Google to fire him.
thankfully, google hires individuals as individuals, and google employees are exceptional and meet the requirements of the job they are hired for, in spite of the average of their community, and in spite of the supposed problems of diversity programs.
It was an internal memo, which someone else leaked.
Whoever leaked that memo was not fired though.
You'd be hard pressed to find a company where leveling personal attacks against your co-workers isn't a fireable offense.
Freedom of speech isn't the same thing as freedom from the consequences of his actions. All it means is you can't be jailed or executed for it. You are not entitled to have people continue to have you in their lives.
Furthermore, his employer has a right to freedom of association, and they have chosen to exercise that right by ceasing to associate with him.
If this is authoritarian, than I will proudly call myself an authoritarian. Perhaps Hobbes was right after all.