> I get that this isn't a unique problem and other forms of "green" power generation also suffer from this such as wind turbines or even photovoltaic panels.

Is there any form of energy storage/generation for which this isn't true? No one every talks about the environmental costs of Lithium batteries, but they're certainly not nil. Pumped storage requires substantial infrastructure as well.