Also, you are painting a false dichotomy between "rich" and "poor". A lot of middle and lower class people can raise healthy and happy children unless they have problems with mental health / violence / substance abuse & so on.

> taking away the only joy in some people's lives

Looking at children as a source of joy instead of a recipient of unconditional care and effort sounds very narcissistic.

Many people find joy in other expensive hobbies, too. The difference is: Doing such a different expensive hobby won't make someone who you produced suffer because you lack the necessary financial backing.

Raising children is a very resource and labour-heavy endeavour. Rich people have more resources and labour. It's only logical that they can afford the children while poor people don't.