They(we?) were more related to each other. I guess that like chimpanzee[1] and bonobos[2], perhaps closer, but my biology knowledge is not strong enough to choose the perfect example.

Note the humans and chimpanzees diverged like 5 million years ago. Neanderthal is very close to us and it's reclassified as it's own specie or as the same specie from time to time.

There is a nice graphic in https://en.wikipedia.org/wiki/Chimpanzee%E2%80%93human_last_... Neanderthal is not in the graphic, but if you want to add it, you should put in near the top left, between us and H. Erectus, probably closer to us.

[1] https://en.wikipedia.org/wiki/Chimpanzee

[2] https://en.wikipedia.org/wiki/Bonobo