So your theory is that civilizations are the way they are because of exogenous rather than endogenous factors? That seems difficult to reconcile with the historical record. Your viewpoint just begs the question. For example, why was Europe in a position to colonize the Americas in the first place? Why weren’t the Spanish greeted by Aztecs with swords and guns?
Why indeed was Europe technologically advanced? Why were the Americas not so much? Resources are one factor, which is why obtaining resources from other lands is valuable. But the main impetus for advancement surely isn't based on one's "skill in advancing". Most people could be trained to fix cars, if desired. Also, Rome fell, but people now live where Rome was with far greater technology. I posit that, if the indigenous peoples of the Americas were given the desire to advance to the level of Europeans, the resources to do so, and time, similar advancement would arise.
You seem to be saying that colonialism advanced society even for the oppressed, but the causality of history is complicated. As far as we know, you may as well say that the extinction of the dinosaurs as it happened was essential for human proliferation. Maybe the dinosaurs would've gone extinct at some point, or diminished greatly, or maybe the dinosaurs and humans would coexist. Just because a somewhat plausible scenario presents itself does not mean it is compelling. You have brought up counterfactuals, so use your imagination seriously, instead of taking the easy way out. If you have a motivating belief on the matter, it is untoward to speak as if you are unbiased and objective.