This is akin to saying "most of the pillaging in history happened by people besides vikings, so you can't really attribute that to them." Yes, others colonized the areas. The Vikings also did during the Viking age.