Let's compromise. So, if he (I'll assume it's "he") concedes there are a lot of shitty fathers out there, can the fact 0 is a valid value in the range (and thus forming the lower bound) just be considered a fact and not a sexist remark? And since some time off is required at the very least to give birth (something a male couldn't do), the lower bound for women would be greater than 0?

Objectively, his bounds make sense. You may not like what that data means, but that in and of itself doesn't make him sexist.