Re:Levels of AGI: Operationalizing Progress on the Path to AGI
by og_kalu on Monday November 13, 2023 @07:05PM
(#38257068)
Computer Chess and Go is so far ahead of human ability now that it doesn't matter how many humans decide to take up the game seriously. I see your broader point but it doesn't really matter here.
Yes what is designated a task is a fair bit arbitrary but game playing AI's have been popularly designated narrow (well basically anything besides LLMs are narrow) for a long time now.
Re:Levels of AGI: Operationalizing Progress on the Path to AGI
by TeMPOraL on Monday November 13, 2023 @07:20PM
(#38257201)
Sure. My point is that this classification is useful only within its category. Say I have a specific problem, like "play chess" or "play StarCraft" or "do maths" or "spellcheck essays". It would be great to have a task-specific list of algorithms ranked by their "Narrow" levels. It would help with use cases like: "oh, model X here spellchecks at level 5N, but is super complex, meanwhile algorithm Y is only Level 3N, but it fits on a napkin; Y is more than sufficient for my MVP". But this rating doesn't let us compare between tasks. That's fine by itself, but then it doesn't make sense to use the same rating for both specialized and general algorithms.
Re:Levels of AGI: Operationalizing Progress on the Path to AGI
by usaar333 on Monday November 13, 2023 @09:41PM
(#38258319)
I don't even understand what the "AI" means in "narrow" AI.
Why is a compiler at Level 0 rather than Level 5?
Re:Levels of AGI: Operationalizing Progress on the Path to AGI
by og_kalu on Tuesday November 14, 2023 @10:00AM
(#38264036)
Compilers aren't thought of as being intelligent. That's why they're level 0
Re:Levels of AGI: Operationalizing Progress on the Path to AGI
by usaar333 on Tuesday November 14, 2023 @11:18AM
(#38265277)
What's intelligence though? GOFAI (level 1) is just logic and search, which are properties of modern optimizing compilers as well.
Even Deep Blue (level 4) is just logic and search.
I mean, they put GPT-4 ("SOTA LLMs") for a subset of tasks at Level 2N, a spell checker at Level 3N, and chess and Go solvers at Levels 4N and 5N. It feels to me that any useful grading scheme would give the opposite ordering. Sure, the current classification meets their definitions - "outperforms X% humans" - but at least with chess and Go, the rating is dominated entirely by the fact that very few people play these games at non-joke level, and the activity itself is super specific, and super narrowly scoped, mathematically speaking. Feels almost like calling CPUs Level 5N at "adding numbers together", since they clearly outperform 100% of humans at that. Or, much less jokingly, one could convincingly argue a PID controller is Level 5 Narrow, because not only it's going to be outperforming 100% of humans at its task, it's also learning and adapting (within its operational envelope).