News for hackers. Stuff that matters.
Re:John Carmack: I’m going to work on artificial general int... by mrfusion on Wednesday November 13, 2019 @08:09PM (#21531746)
I don’t think there are many researchers in AGI. AFAIK it’s kind of a joke field because no one has any clue how to approach true AGI.

Please correct me if I’m wrong.

Re:John Carmack: I’m going to work on artificial general int... by Jach on Wednesday November 13, 2019 @09:20PM (#21532210)
People have approaches. There's no end to half-assed "I thought about this for 10 seconds, how hard could it be!" solutions, really old approaches from decades ago where the brightest academics thought they could lick the problem over a summer, and some new public or hidden approaches that might be promising but (I can't know of course) I predict will still look a lot different than the final thing.

I think a big reason there are few in AGI is due to PR success from the Machine Intelligence Research Institute and friends. They make a good case that things are unlikely to end well for us humans if there's actually a serious attempt at AGI now that proves successful without having solved or mitigated the alignment problem first.

Re:John Carmack: I’m going to work on artificial general int... by someguyorother on Thursday November 14, 2019 @07:32AM (#21534884)
MIRI's concerns are vastly overrated IMHO. Any AGI that's intelligent enough to misinterpret its goals to mean "destroy humanity" is also intelligent enough to wirehead itself. Since wireheading is easier than destroying humanity, it's unlikely that AGI will destroy humanity.

Trying to make the AGI's sensors wirehead-proof is the exact same problem as trying to make the AGI's objective function align properly with human desires. In both cases, it's a matter of either limiting or outsmarting an intelligence that's (presumably) going to become much more intelligent than humans.

Hutter wrote some papers on avoiding the wireheading problem, and other people have written papers on making the AGI learn values itself so that it won't be tempted to wirehead. I wouldn't be surprised if both also mitigate the alignment problem, due to the equivalence between the two.

Re:John Carmack: I’m going to work on artificial general int... by TeamSlytherin on Thursday November 14, 2019 @01:54AM (#21533360)
Yes, AGI is as much or more cognitive neuroscience and philosophy than computer science right now, but a lot depends on the approach one is taking. It's funny to think you have some kind of working model you can throw research data against to see how it holds up, and then doubt yourself when you spend 3 hours on Twitter arguing over fundamentals with another person that is also convinced of their model. A lot of popular ideas sound crazy (or non-workable), so you just have to accept that whatever idea you are pushing is going to crazy as well.
Re:John Carmack: I’m going to work on artificial general int... by MetalGuru on Thursday November 14, 2019 @01:19AM (#21533218)
The alignment problem?
Re:John Carmack: I’m going to work on artificial general int... by retsibsi on Thursday November 14, 2019 @03:39AM (#21533826)
> The alignment problem?

The problem of ensuring that the AI's values are aligned with ours. One big fear is that an AI will very effectively pursue the goals we give it, but unless we define those goals (and/or the method by which it modifies and creates its own goals) perfectly -- including all sorts of constraints that a human would take for granted, and others that are just really hard to define precisely -- we might get something very different from what we actually wanted.

Re:John Carmack: I’m going to work on artificial general int... by tim333 on Thursday November 14, 2019 @04:29PM (#21539630)
Wikipedia:

>A 2017 survey of AGI categorized forty-five known "active R&D projects" that explicitly or implicitly (through published research) research AGI, with the largest three being DeepMind, the Human Brain Project, and OpenAI.

Hassabis and DeepMind have a fairly organised approach of looking at how real brains work and trying to model different problems like Atari games then Go and recently Starcraft. Not quite sure what's next up.