The number being that low is suspicious. In my vague, personal experience, the upper tiers of computing are about 10% female. The odds of 1 in 23 are 0.315. Plausible, but worth investigating.

Are the speaker selectors favoring men? That should be possible to test for. If so, it needs to get clamped down on hard.

Are women less inclined to try to speak at these conferences? If so, why? It's likely that making presenting more female-friendly would be worth it.

Is this subfield just that male? If so, it's not the conference-organizers' place to deal with it, but someone should sniff around for active biases.

In short, this is suspicious enough to justify a real investigation, and I'd be very interested to read the results of one.