Exactly. I grew up in quite poor family and shared room/toys with my sister the whole time - we played the same Lego sets, I made dresses from cloth scraps for dolls, we both collected toy cars and Kinder sets. Still, since the early childhood the differences were already clear and the gap only increased with the time. I strongly believe that the key reason on why we have few women in IT (or math, physics, engineering for that matter) is natural preferences combined with peer pressure, so simply making childhood sex neutral won't make a real difference.
I don't think we can say either way what it is that "keeps" girls from science, but I honestly lean towards the preference angle. There are differences between males and females and that's okay.