It's also possible that Neanderthals prevented modern humans from leaving Africa until their populations were already in decline for other reasons (disease? inbreeding? infighting? overhunting?). In that case we may have just sped up the inevitable.