News for hackers. Stuff that matters.

A comment on Show HN: ARC Prize – a $1M+ competition towards open AGI progress. Parent comment

Re:Show HN: ARC Prize – a $1M+ competition towards open AGI ... by llm_trw on Wednesday June 12, 2024 @02:57AM (#40655367)
Babies, unlike machine learning models, aren't placed in limbo when they aren't running back propagation.

Babies need few examples for complex tasks because they get constant infinitely complex examples on tasks which are used for transfer learning.

Current models take a nuclear reactors worth of power to run back prop on top of a small countries GDP worth of hardware.

They are _not_ going to generalize to AGI because we can't afford to run them.

[ Parent | on HN ]
Re:Show HN: ARC Prize – a $1M+ competition towards open AGI ... by larodi on Wednesday June 12, 2024 @07:14AM (#40656755)
> Current models take a nuclear reactors worth of power to run back prop on top of a small countries GDP worth of hardware.

Nice one. Perhaps we are to conclude the whole transformer architecture is amazingly overblown in storage/computation costs.

AGI or not, we need better approach to what transformers are doing.

[ Parent | on HN ]