Men behave differently around women, compared to an all-male environment. It's mostly subconscious, but men (especially the older generation) would instinctively put up a tough front while women are present, and would be hesitant to show signs of weakness, so some would have trouble truly opening up. No doubt there are also male health concerns that men would feel more comfortable talking about to other men.

It's not sexist to want men only spaces, just like women only gyms and salons are not sexist.