We also don't call, say, the Spanish conquest of the Netherlands "colonialism", yet we still celebrate our independence from Spain (indeed, our entire royal house is based on this).

I think it's mostly just a bit of a bikeshed over terminology to be honest; we could also call it "European conquest of Africa", or whatnot, but that's just the same as "colonialism" with a different name.