Under no reasonable time frame for human existence is his claim even close to valid.
The fact is, we've been coming out of an ice age for the past several millennia, and life has generally gotten better as a result. The rate at which that process happens is very important, though. If Dyson did overlook any important counterpoints, that seems like a big one.
We know exactly what the Arctic is going to look like and its impacts. We have known and are observing the melting of glaciers in the Alps, Andes and Himalayas to catastrophic effects in downstream environments.
Anything is subject to debunking if you're unwilling to look at the most sophisticated evidence available. Unfortunately, there is no model that will tell you that a 4 degree increase in average temperature won't end most of human society as we know it.
Moreover, “Contrary to popular accounts, very few scientists in the world – possibly none – have a sufficiently thorough, 'big picture' understanding of the climate system to be relied upon for a prediction of the magnitude of global warming. To the public, we all might seem like experts, but the vast majority of us work on only a small portion of the problem.” (Roy Spencer)
I'm sorry but I'm not gonna take seriously people who want to criticize model sensitive as a reason to ignore climate change when the current glacial melt, the ongoing ocean acidification and all the phenomena that have already been unleashed by the existing levels of climate change continue unabated.
Small consolation if you live in one of the former places, of course. But he's not wrong. Earth as a whole is a bit colder than it would be in the best of all possible universes.
Other effects like ocean acidification are harder to hand-wave away. I hate cold weather, myself, but it's not just about terrestrial climate.