News for hackers. Stuff that matters.
Re:It takes a lot of energy for machines to learn by firebaze on Wednesday December 30, 2020 @03:55PM (#25585984)
This is beyond ridiculous, even if compared to the energy to train a human mind at the age of, let's say, 35. Raise a child to the age of 35 and the spent energy will surpass the mentioned energy cost by at least 2 orders of magnitude.

And no, I don't ridicule AI ethics researchers: the energy cost of AI is negligible to the serious issues. The real harm stems from other areas.

Re:It takes a lot of energy for machines to learn by MAXPOOL on Wednesday December 30, 2020 @04:37PM (#25586441)
Human mind (aka brain) has consumed little over 6000 kWh energy when it's 35.

From the paper they cite

    algorithm    kWh-PUE
    ----------   -------
    ELMo             275
    BERT(base)     1,507 
    NAS          656,347  
NAS was Transformer (213M parameters) for neural architecture search 2019. They didn't have kWh numbers for GPT-2.
Re:It takes a lot of energy for machines to learn by sbierwagen on Wednesday December 30, 2020 @05:28PM (#25586965)
I don't know about that 6000 KWh number.

More importantly, a human brain isn't trained from scratch at birth. It's had a few hundred million years of pretraining.

Re:It takes a lot of energy for machines to learn by wongarsu on Wednesday December 30, 2020 @05:54PM (#25587257)
The 6000kWh number is assuming 20W over 35 years. That might be right if you only count the brain, but then you would have to revise the neural network numbers to only count CPU and memory, which would easily halve those numbers (fans take a lot of power).

If we want to account for power supply, temperature control etc. we could use the entire calorie count of a human over 35 years, leaving us with around 30000kWh (assuming a health, not-overweight human). You could now argue that that's too high (the human is doing a lot of other things). But as you pointed out it's a bit of a pointless comparison anyways as humans don't start from scratch.

Re:It takes a lot of energy for machines to learn by antisthenes on Wednesday December 30, 2020 @07:58PM (#25588318)
> we could use the entire calorie count of a human over 35 years, leaving us with around 30000kWh (assuming a health, not-overweight human)

That's still too low, considering you can't just have a human eating food and educating themselves in a vacuum.

We need to account for energy going to and from school, shelter, heating, clothing. Basically all the necessities that enable someone to learn (and their professors)

That's another 5x at a minimum.

Re:It takes a lot of energy for machines to learn by sifar on Thursday December 31, 2020 @02:03AM (#25590304)
For the NN architectures, you'll have to factor in there production, design, the energy of the humans involved and so on and on.

This doesn't take you anywhere.

Re:It takes a lot of energy for machines to learn by goatlover on Wednesday December 30, 2020 @03:59PM (#25586034)
But that person would be learning tons of other things as well. It’s not equivalent.
Re:It takes a lot of energy for machines to learn by firebaze on Wednesday December 30, 2020 @04:00PM (#25586048)
That person would probably be one of a billion. The AI is singular. This should cancel out.