I don't think it would be catastrophic, unless these children spend 90% of their time playing VR games. I would expect it to be similar to exposing a child to a second language. If you replace their exposure to the previous language completely with the new one, their proficiency at the first one will suffer but if you provide enough time for both, the brain will improve in both.