You think the authors Damore cited rejected him. Perhaps you have an example of that. I haven't seen anything beyond the authors agreeing with Damore's interpretation of the facts and saying they don't know about the conclusions. It's obviously not very compelling given that Damore was metaphorically tarred on feathered on social media and fired, so researchers saying they "don't know" about the conclusions might be because they genuinely don't know or might be because they don't want to invite backlash of their own.
I cannot personally derive a conclusion about Damore from the literature. What I can do is ask world experts about their thoughts. In general, this is an effective way of developing reasonable opinions about the state of research. I'm friends with a few social psych faculty who study workplaces and let's just say that they don't hold Damore in high regard.
I'm super uninterested in having an argument about the merits of his essay since that discussion has been had thousands of times and no progress can be made. But "well he has a master's degree from harvard" is among the least compelling arguments in his favor.
If world experts have weighed in on Damore, link to their opinions. I'm happy to discuss those.