Racism doesn't really figure into regular Freemasonry either, as it was founded as a universalist enlightenment society. When you look at who has historically made a point of suppressing them, it's worth asking who the good guys really were. They're a reliable canary for civil liberty. Prince Hall masonry, founded by and for African American men is still going strong in the US, and it is in concordance with most north american grand lodges. The main thing is you can't be an atheist in regular masonry, which is analogous to having a policy against racism in that atheism is traditionally also viewed as an obtuse and unnecessarily irreconcilable belief that can call someones moral judgment into question. The continental/european stuff is different, and much of it isn't connected to the mainstream anglosphere masonic culture.
Why must this be male only to be private?
Freemasonry is interesting, in that it actually had a schism in part over exclusionary membership clauses around 1877 (Regular vs Continental). Although apparently, masonry itself forbids discussion of religion or politics during its meetings.
But much of the early lodge structure and exclusion has since been rewritten and broadened, as the majority of them were founded in the 1870s - 1910s, when prevailing social attitudes were themselves much more sexist and racist.
(Interesting, a majority were operated for the purpose of effectively providing their membership a social safety net and insurance, both of which were lacking in government terms at the time!)