The parent was making a joke, but one way to answer your question might be:

You feel bad because you assume they - like you - feel something when you mistreat them, and you assume this at a very deep level that might not, itself, feel like anything to you.

While some humour was intended, I do actually grapple with this question. The answer that rings truest for me, especially as a newly minted father, is Vonnegut’s: “A purpose of life, no matter who is controlling it, is to love whoever is around to be loved.”