My fart has an impact on climate change too. Doesn't mean it's meaningful. Same for these models.

Yup, kind of sucks but the end result is likely everyone and every actual living thing will suffer from this wave of “AI” because it’s an arms race now, burn it up.

Here is your answer to the fossil fuel problem.

https://www.youtube.com/watch?v=kpeODvGJE1Q

update: found a better one.

The answer already exists but people refuse to implement it.

We have nuclear, but a lot of Americans are concerned about nuclear for reasonable reasons.

We have renewables, but those can't take up the majority capacity of a grid unless we start adding massive batteries.

Then there's grid rebalancing where we incentivize people to use and store renewable energy locally, this lessening the strain on the grid but that still results in fossil fuels or nuclear.

Hydro has been found to be environmentally destructive. I'm not sure if that was just my state or if that's ubiquitous.

What were you thinking?

Probably just salivating over Dyson spheres, or whatever quantum woo-woo showed up a few news cycles ago about energy appearing "from nowhere"...

Nuclear + solar panels everywhere

Ah, so people don't just "refuse to implement it", yeah? There's been a large number of disasters with long-term consequences and the industry is claiming it has more reliable systems, which is also what they said before many of these nuclear disasters occurred.

I'm hopeful for nuclear, but they have a ways to go in proving themselves to the wider public. With that said, I think you could've positioned what you said a bit more fairly.

It's one of the tired tropes that gets brought up every time AI/ML is brought up.

Everything we do has climate change impact. Power consumption is among the ones that's easiest to get "green", and there is significant progress specifically from cloud operators (at least Google, I assume others are similar).

https://www.reddit.com/r/MachineLearning/comments/hwfjej/d_t... suggests hundreds of GPU-years for training such a model under optimal assumptions, so let's assume thousands for the new models under real assumptions.

https://images.nvidia.com/content/technologies/volta/pdf/tes... says 300 W.

3000 GPU-years at 300 W = 7.9 million kWh. This would assume that they used those GPUs and not more efficient accelerators.

https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 says 0.855 pounds of CO2 emissions per kWh. That's 388 g in normal units, and in line with what other countries are reporting (Germany was 420 g/kWh). g/kWh = metric tons per million kWh. This assumes the data centers are not using "greener" than average power.

So training such a model produces ~3000 metric tons of CO2 emissions under the assumption of 3000 GPU-years (how convenient). A round trip from SFO to London in Economy is roughly 0.9 metric tons. (https://www.icao.int/environmental-protection/Carbonoffset/P... Building a house (of unfortunately unspecified size) is 15 to 100 tons. (https://climate.mit.edu/ask-mit/how-much-co2-emitted-buildin...)

So while building one of these models does have an impact, it's on the order of other common activities that benefit far fewer people. Because once trained, those models provide values to millions of users.

Going back to the flights example, training one such massive model is likely about as bad as one larger research conference, once you consider the impact of hotels etc.

IMO the demands to justify the impact are ridiculous, coming from people who are just looking for any excuse to criticize, on par with demanding that any researcher doing any research justifies the carbon footprint of their commute as part of their research paper. Thus, it's a good thing that they didn't waste time and space in their paper addressing those claims, and we only see an early, commented out section that's equivalent to "TODO: Should we address these claims that keep getting thrown?"

I understand your view point, but I guess your argument really boils down to, it's all justifiable in some way or another.

I don't disagree with you either, I live in a foreign country and fly home to see family each year, so I'm just as bad as everything I'm claiming is bad.

I guess this proves my point though, it's all justifiable to some degree: "I have to go see my family.", "this benefits millions." and on it goes.

> Because once trained, those models provide values to millions of users.

Do they? Those of us in tech often take the positive value of technological progress as a given, but when looking at, e.g., the example in OpenAI's paper of GPT4 tricking a human into solving a CAPTCHA, I think it's quite clear there's possible negative value as well, and claims about "value to millions of users" probably need to be substantiated.