The first smallpox epidemic alone killed about half of the Incas.
To put this in perspective, this period came right after 100 years of bloody civil war in Spain that produced some of the most efficient killers the world had ever seen.
[EDIT] I shouldn't have said that they weren't Europeans, but that not all of Europe was represented, only Portugal and Spain. That blaming Poland or Germany for the activities of the Spanish was pretty unfair.
Edit: also worth pointing out that the Spanish monarchs for the period we're talking about were the House of Hapsburg and hence claimed extensive territories in what's now Germany, Austria and Holland, so even many land-locked European regions were involved. There are a surprising number of Iberianized German names in colonial documents from Latin America.
Bizarrely, there was even a minuscule colonial empire ruled by a tiny vassal state of Polish-Lithuanian common wealth for awhile: https://en.wikipedia.org/wiki/Duchy_of_Courland_and_Semigall...