That money doesn't come for free. It's perfectly rational for people who don't directly benefit from a lot of visitors--but are inconvenienced by them--to wish to forgo both the visitors and the money they spend/taxes they bring in.