Not sure about justification, but there's evidence that their brains were focused more on vision and movement.

https://www.seeker.com/neanderthals-lacked-social-skills-176...

Although, it appears that Neanderthals were bigger, but shorter than modern humans. I thought they were taller as well, but apparently that's not true