Just a few things on your points, men are still financially responsible for children they have raised, even if they are not genetically related. Most people think it's the right thing, too.

Infidelity is not illegal either, by the way, there's no penalty for it because the state doesn't have a say in who sleeps with who. Of course it's a cause for divorce though.

In the end, no matter what might happen to the parents, the justice system is all about the interests of the children. It might not always be fair to the father or the mother, but if it's better for the kid then that's how it is.

If infidelity is a valid cause for divorce, the state acknowledges that there's something wrong with it.

Moreover, actively tricking somebody into taking on the social obligations of fatherhood, is a dramatic violation of most major moral obligations that humans have. It also violate the child's right to know both parents.

Aside, there is an easy way out of this conundrum: mandatory paternity test at birth.

>Most people think it's the right thing, too

You only make the appeal to popularity when the public holds the same opinion as you.

> Of course it's a cause for divorce though

If your partner can divorce you for infidelity, that is a penalty that they can selectively enforce. When the state makes it a justification for divorce, the implicit message is that infidelity is bad for marriages. Why, then, would the state want to reduce the penalties for infidelity by making the husband of a cheating wife financially liable regardless of paternity?

>the justice system is all about the interests of the children

If the state cares about the welfare of a child the state can give welfare payments to the mother.

Just because the system is unfair to some parties doesn't mean it's done in the best interest of one party. I don't buy it that the state is motivated here by the welfare of children. Marriage, and more generally being raised by two parents, is quite beneficial to children. If the state acted in accordance with this, you would expect the state to discourage (increase the costs) of divorce rather than encourage it (make it easier or reduce the costs). Here we are seeing the opposite.