Hormones mean women are less strong and thus less able to do the job. You get more work out of males.

Plus women were more suited for sex, these were not enlightened times so rape of slave women was not frowned on.