News for hackers. Stuff that matters.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by wcerfgba on Friday August 19, 2022 @06:25PM (#32526750)
Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence' unless it has a similar embodiment to us, able to move around a physical environment with it's own limbs, sense of touch, sight, etc. ? Any arguments against the necessity of this? I feel like any AGI developed in silico without freedom of movement will be fundamentally incomprehensible to us as embodied humans.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ml_basics on Friday August 19, 2022 @06:38PM (#32526876)
I don't think it is necessary - though all forms of intelligence we're aware of have bodies, that's just a fact about the hardware we run on.

It seems plausible to me that we could create forms of intelligence that only run on a computer and have no bodies. I agree that we might find it difficult to recognize them as intelligent though because we're so conditioned to thinking of intelligence as embodied.

An interesting thought experiment: suppose we create intelligences that are highly connected to one another and the whole internet through fast high bandwidth connections, and have effectively infinite memory. Would such intelligences think they were handicapped compared to us because they leak physical bodies? I'm not so sure!

Re:John Carmack's new AGI company, Keen Technologies, has ra... by wcerfgba on Friday August 19, 2022 @07:41PM (#32527445)
not necessarily handicapped but certainly different. i did an artwork about the inherent problems with an intelligence that was not aware about how other entities would perceive it https://jpreston.xyz/i-can-feel.html
Re:John Carmack's new AGI company, Keen Technologies, has ra... by ericb on Friday August 19, 2022 @06:43PM (#32526934)
With VR, you can get the sight and physical environment you mention. That seems like, at minimum, proof that in-silico intelligence won't be blocked by that requirement.

I do fully agree that any intelligence may not be human-like, though. In fact, I imagine it would seem very cold, calculating, amoral, and manipulative. Our prohibition against that type of behavior depends on a social evolution it won't have experienced.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by PaulDavisThe1st on Friday August 19, 2022 @06:54PM (#32527039)
> With VR, you can get the sight and physical environment you mention

"physical environment" ? VR lets you operate on and get sensory input based on a fairly significantly degraded version of physical reality.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by jamiek88 on Friday August 19, 2022 @07:27PM (#32527343)
Also relies on quirks and tricks of human sight and cognition to make sense!
Re:John Carmack's new AGI company, Keen Technologies, has ra... by turbohz on Friday August 19, 2022 @07:26PM (#32527338)
Are you suggesting that all reality sensory input is equally relevant?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by PaulDavisThe1st on Friday August 19, 2022 @08:35PM (#32527798)
The opposite. VR is an impoverished reality. Any AI seeking to do science, for example, would rapidly hit "bottom".
Re:John Carmack's new AGI company, Keen Technologies, has ra... by adamsmith143 on Friday August 19, 2022 @06:48PM (#32526977)
>Does anyone else subscribe to the idea that AGI is impossible/unlikely without 'embodied cognition', i.e. we cannot create a human-like 'intelligence'

Seems like you are confusing Consciousness with Intelligence? It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious.

>I feel like any AGI developed in silico without freedom of movement will be fundamentally incomprehensible to us as embodied humans.

An AGI will be defacto incomprehensible to Humans. Being developed in Silicon will have little bearing on that fact.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by aryanmann on Friday August 19, 2022 @07:54PM (#32527535)
Good point. The lines between consciousness and knowledge seem blurred too since though we may have certain kinds of knowledge without embodiment (such as 2+2=4), other knowledge such as qualia [1] may be inaccessible to an unembodied agent. The classic example of "Mary knowing everything about the color red before having seen the color red physically. Does she still learn more about the color red when she actually experiences it with her own eyes?"

[1] https://plato.stanford.edu/entries/qualia/

Re:John Carmack's new AGI company, Keen Technologies, has ra... by sph on Saturday August 20, 2022 @01:21AM (#32529335)
> It's completely plausible that we will create a system with Intelligence that far outstrips ours while being completely un-Conscious.

Is it? Is there proof that consciousness is not a requirement to close-to-human level intelligence?

Given that we do not even know how consciousness works in our brains, I don't think we have the answer yet. I'm not sure it's plausible to make such an assumption.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by adamsmith143 on Saturday August 20, 2022 @11:57PM (#32537285)
>Given that we do not even know how consciousness works in our brains, I don't think we have the answer yet. I'm not sure it's plausible to make such an assumption.

Possibly a fair argument, but I think the opposite would be just as valid then. That you can't assume that consciousness IS a requirement for intelligence.

It seems hard to argue that you don't need some intelligence for consciousness, there seems to be a kind of minimum. We can already see that narrow intelligences for Vision, NLP, game playing, etc are at or above human level and no one claims these systems are conscious. So why should we assume that some meta system of these already existing systems would be?

Re:John Carmack's new AGI company, Keen Technologies, has ra... by darawk on Friday August 19, 2022 @08:22PM (#32527709)
I think the burden of argumentation is on the person proposing the constraint. I can only think of one argument for why embodiment might be necessary, and that is our only existence proofs for AGI (humans, maybe some animals) are embodied. But those existence proofs also share a bunch of other properties that probably aren't particularly relevant to AGI. Being carbon-based, having DNA, breathing oxygen, etc.

Why do you think embodiment should be necessary?

Re:John Carmack's new AGI company, Keen Technologies, has ra... by abecedarius on Friday August 19, 2022 @08:56PM (#32527941)
> Any arguments against the necessity of this?

Supposing it's needed, a universal computer can simulate physics. Now we're just haggling about the price.

It's possible that developing a robot is the most fruitful angle to take on the problem -- giving you the right subproblems in the right order, with economic value along the way. Hans Moravec thought so, writing a few decades back.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by lambdatronics on Saturday August 20, 2022 @12:28PM (#32533215)
If we want the intelligence to be recognizably human-like, then yes. As to whether the AGI could be trained in a simulation vs needing to really be physically embodied: I used to believe not, but then I learned about sim2real. (Ex: [0]) Training blank-slate deep learning algorithms on real hardware is prohibitive. It requires millions of hours of experience. Those hours are really expensive on current-day hardware (also cannot be accelerated, stuck with wall clock time). But if pre-training in simulations can be effective, then I think we have a decent shot in the near-term.

Human-level and human-like are not necessarily the same though. I doubt human intelligence is as general as we like to think it is. There's probably a lot about what we consider intelligence that is domain-specific. Training AGI on unfamiliar domains could be super valuable because it would be easier to surpass human effectiveness there.

My pet conjecture, though, is this: the ability to experiment is key to developing AGI. Passively consuming data makes it much harder to establish cause and effect. We create and discard hypotheses about potential causal links based on observed coincidences (ie, small samples) all the time. Doing experiments to confirm/refute these hypotheses is much less computationally expensive than doing passive causal inference, which enables us to consider many more potential causal linkages. That allows us to lower the statistical threshold for considering a potential causal relationship, which makes it more likely for us to find the linkages that do exist. The benefit of developing accurate causal models is that they are much more compact & computationally-effecient than trying to model the entire universe with a single probability distribution.

[0] https://openai.com/blog/solving-rubiks-cube/

Re:John Carmack's new AGI company, Keen Technologies, has ra... by gushogg-blake on Saturday August 20, 2022 @05:16PM (#32535196)
Yes! I have a feeling that as you define the goal of "something that has human-like intelligence but isn't human" more clearly, the less sense it will make because it will become clear that our idea of intelligence depends heavily on the specific ways that humans relate to the world. I wrote a blog post on this - https://gushogg-blake.com/posts/agi
Re:John Carmack's new AGI company, Keen Technologies, has ra... by the8472 on Friday August 19, 2022 @07:36PM (#32527412)
The term AGI means "artificial general intelligence", it does not mean human-like intelligence. So it being incomprehensible to us does not preclude it from being an AGI.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by Ekaros on Friday August 19, 2022 @06:32PM (#32526825)
Human like is actually very good question. It doesn't even come to embodiment, but in general how it would think and act. Lot of how we do is due to cultural, language, training and education.

For example, which language would it "think" in? English? Some other? Something it's own? How would it formulate things? Based on what philosophical framework or similar? What about math? General reasoning? Then cultural norms and communication?

Re:John Carmack's new AGI company, Keen Technologies, has ra... by PaulDavisThe1st on Friday August 19, 2022 @06:55PM (#32527045)
The argument is that every question in your post is a red-herring. That is, humans don't actually function in the way that your questions suggest, even if they have the "experience" of doing so.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by copperx on Friday August 19, 2022 @07:36PM (#32527413)
I do expect it if we want AGI to have appropriate emotions. We know emotions are sometimes created by cognition, and often created by chemical processes. This will have to be incorporated into the model somehow if we want to have human-like intelligence.

However, AGI doesn't need embodiment in order to be useful. In fact, it might be detrimental. I believe AGI will happen much sooner than embodied AGI.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by makeitdouble on Friday August 19, 2022 @06:57PM (#32527069)
I see that as two separate goals.

One is to build something inteligent (an AGI), and the other is something human like. Intuitively we could hit the AGI goal first and aim for human like after that if we feel like it.

In the past, human like inteligent seemed more approachable for I think mostly emotional reasons, but for our current trajectory if we get anything inteligent we’d still have reach a huge milestone IMO.

Re:John Carmack's new AGI company, Keen Technologies, has ra... by oefnak on Friday August 19, 2022 @06:38PM (#32526883)
When you're sitting behind your computer for a while, you can forget it is there, and just 'live' in the internet, right? That's not so big a difference maybe, if you can abstract the medium away that feeds you the information.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by imposter on Saturday August 20, 2022 @04:33PM (#32534969)
What's the difference between moving around and picking up a document, to say hitting an API endpoint to fetch the content of the same document?
Re:John Carmack's new AGI company, Keen Technologies, has ra... by wibblewobble124 on Saturday August 20, 2022 @05:26PM (#32535262)
Human and animals interaction is all physical. Babies and early children spend most of their brain development on physicality. By depriving a mind of having a body, that mind cannot have empathy for the human condition or animals, the idea of living in a body will only be abstract. Have you seen the movie Johnny Got His Gun? It’s one big statement that being disembodied is hell.
Re:John Carmack's new AGI company, Keen Technologies, has ra... by mbrochh on Sunday August 21, 2022 @11:45PM (#32547097)
You are 100% correct, which means that whatever we will create, will be absolutely alien to us (and probably kill us).
Re:John Carmack's new AGI company, Keen Technologies, has ra... by pfd1986 on Saturday August 20, 2022 @07:30AM (#32531192)
this point is brought again and again by the late John Holland (check out his books) and by his protege Melanie Mitchel...