Because changes in climate mean that the existing infrastructure (in the form of cities, farmland, etc) that are optimized for the current conditions are no longer optimal. Very few cities are designed for conditions below sea level or in a desert, but if the city already exists and the desert and oceans comes to where they are then bad things will happen.

I mean you're right that conditions in some places might get better, but that doesn't help if people have to build the infrastructure there to support a modern settlement in order to take advantage of it. When people talk about the cost of climate change, they're taking about the costs of building or adapting infrastructure.