If that's really a worry, invite 3 women. That will make it clear that it's not a date, and it'll be much more likely that they'll feel safe talking about gender issues if they know they'll have backup and confirmation.