Dosage matters. Many things that are completely destructive or toxic are not a problem and even beneficial and enriching in small dosage. Tourism is most definitely one of those things. Typically it benefits a very few in the host location while externalizing the destruction of the culture, the environment and the quality of life for the many that have these places as their home. So it is not about 'you' vs 'the others'. It is about the quantity of people exceeding what is a healthy balance.