You don't want everyone to travel - many people don't have a sense of respect and "light foot" that it takes to travel to foreign places without degrading or damaging them.
Not everyone wants to (Openness), or could reasonably do a good job of respecting the host culture and environment (Agreeableness and Conscientiousness).
I, for one, will never vote for making my flights more expensive
For example, say you added a $X / mile flight tax that collected $50 billion dollars in a year. You then take that money and divide it up evenly among all tax payers. Around 200 million people file taxes each year, so that would be a $250 refund to each filer. For people who never fly, that's free money. For people who fly infrequently, they probably break even. People who fly all the time pay a lot more. You reward non-flyers and penalize frequent flyers.
But you're right. We're all selfish. I want you would stop flying because it's bad for the climate but I'm not going to stop because my personal utility from flying is very high.