A better argument would be that every time brain differences were found, they turn out to be irrelevant. However, I'm not sure even that is true, considering some colleges are experimenting with mental rotation training to help female engineering students, with good results: https://www.kqed.org/mindshift/43802/can-teaching-spatial-sk... That example also shows identifying gender differences is not a bad thing; it allows coming up with ways to compensate.
But the plasticity means that cultural practises are likely to have an effect on brains, in the same way that London Cabbies famously have over developed hippocampus with more nerve cells than they had before due to the requirements of their jobs.
Finding brain differences does not, therefore, mean that the differences are genetic or directly gender related, but could be culturally generated through different roles being adopted for social reasons.