You are allowed to and should grieve for that child. There's therapy based on attachment theory where you learn to how to be the parent to that (inner) child.