That's a good point about renewable energy sources.
I'd still argue it's reasonable to assume that ML will consume more power as use-cases for it grow (i.e. more and more models are trained), and, therefore, I don't think it's unreasonable to consider the energy usage.
Honestly - I think what you may have heard is "ML is bad/evil." And yeah some folks probably feel/vibe that.
That's different from "Like any other technology the full context of ML deserves to be considered, and since it's so hyped by billion-dollar companies right now, maybe a bit of pushback on all the hype isn't a terrible idea."
And yeah we should definitely look at the airline industry; pretty sure Mother Jones does, and there's no reason we can't look at the energy impact of multiple industries.
Likewise, yes, language models use energy, but objectively, it's not that much energy and the sources for the energy are sometimes green and always efficient (i.e. Coming from the grid and not by burning jet fuel).
You can't really solve the problem of lung cancer by addressing scented candles. Reducing energy used by language models likewise won't have an effect.
There are hundreds of thousands of flights per day. Adding an additional flight, or even an additional thousand flights, to substantially improve Google doesn't seem like a big cost. Consider: Would your life be worse if one random trans-American flight was cancelled today, or if Google searches became 10% worse?
Another way of thinking about this same point, is, why is the author writing about language models if they are so concerned about the environment. Surely the airline industry is a better subject as, again, they fly hundreds of thousands of flights each day. It's hard to take someone seriously when they are focusing on an infinitesimal part of a huge problem.
It's also misleading because there is a difference between energy used by an airline flight, where the energy comes from burning jet fuel, and energy used in a data center. In Google's case (the people training BERT) the energy used was 100% renewable - Google reached that goal in 2017. Perhaps Open AI didn't use renewable energy to train GPT-3, but I wager they didn't power their machines by burning jet fuel either.
Maybe electricity used to power to train language models will become a meaningful issue at some point in the future. I don't think that future is close at hand though.