News for hackers. Stuff that matters.
Re:John Carmack and Rich Sutton partner to accelerate develo... by garblegarble on Saturday October 07, 2023 @08:35AM (#37801202)
>I wondered the other day what the result would be if you strapped a microphone on a baby and every utterance that both the baby and mic heard was fed through speech-to-text and went toward training the baby's personal LLM.

I can't find the source for this, but I read about a rather unpleasant animal study in cats where they took two kittens. One was free to look around, but the other they immobilised in some way, and made it see what the other kitten saw as it looked around. They discovered the immobilised kitten did not develop normally whereas the mobile kitten did, suggesting that it's not just the sensory input, but it being feedback to some internal agency within the brain. I'll try to have another look to find the study later and update this post if I can.

I imagine that the same core insight may be true for attempts to develop AGI by giving them an audio-visual copy of a human's environment growing up: that internal state driving (and getting feedback from) the action to investigate/interact with the world is key.

Re:John Carmack and Rich Sutton partner to accelerate develo... by spockz on Saturday October 07, 2023 @10:14AM (#37801875)
I suppose a large part of learning is not just seeing things happen but also deciding to do something, executing on it, and then feeling the result. So not surprising.
Re:John Carmack and Rich Sutton partner to accelerate develo... by AJRF on Saturday October 07, 2023 @11:15AM (#37802405)
Curious to know how you found this paper? I'd like to be able to find interesting papers but I've not found a place of curation unfortunately.
Re:John Carmack and Rich Sutton partner to accelerate develo... by garblegarble on Saturday October 07, 2023 @12:04PM (#37802843)
I'm afraid I don't remember how I found this paper, but it might be from following Dr Abeba Birhane[1], she's an anti-AI academic (which is why I followed her originally, to get reality checks / contrasting viewpoints), but she also posts and boosts a lot of interesting cognitive science (her field). I also follow Olivia Guest[2] for the same reason.

Many of the papers go over my head at times of course (despite my interest, I'm firmly a layman!)

More generally, I've had a lot of success with searching for meta-analysis papers on topics I'm interested in, since the authors have already done the hard work of researching the literature and presenting core takeaways (as well as providing terms for more searches).

1: https://bsky.app/profile/abeba.bsky.social or https://twitter.com/Abebab

2: https://bsky.app/profile/olivia.science or https://twitter.com/o_guest

Re:John Carmack and Rich Sutton partner to accelerate develo... by AJRF on Saturday October 07, 2023 @01:39PM (#37803765)
thank you for taking the time to answer and to provide links, appreciate it.
Re:John Carmack and Rich Sutton partner to accelerate develo... by romesc on Saturday October 07, 2023 @01:25PM (#37803625)
For what it's worth, this was a paper I was exposed to during grad school for robotics/CS. I'm not saying one has to go to school to get good curation of content, but it is certainly one way to be exposed to a well structured foundation of knowledge in your field.
Re:John Carmack and Rich Sutton partner to accelerate develo... by jedberg on Saturday October 07, 2023 @01:23PM (#37803599)
This isn't a very new concept -- I learned about this in the 90s. Even then the language researchers knew that interaction with the environment was crucial for learning in humans. There were some natural experiments where kids were born without limbs for example and had trouble with early language acquisition because they couldn't experience the environment the same way as most others.

And the AI folks argued even then that the only way to create a language model that truly "understands" would be to put it in a robot that can experience the world in the same way as a human.

Re:John Carmack and Rich Sutton partner to accelerate develo... by danielbarla on Saturday October 07, 2023 @01:25PM (#37803623)
> They discovered the immobilised kitten's visual processing did not develop normally whereas the mobile kitten's did, suggesting that it's not just the sensory input, but it being feedback to some internal agency within the brain.

> I suspect the same thing would be true for attempts to develop AGI...

On the other hand, I would not be surprised if this did _not_ generalize and translate to LLMs. Biological entities on earth tend to have tons of fairly arbitrary developmental legacy which may or may not translate well to an artificial one. In the experiment, they didn't make billions of clones of the cat with a variety of different parameters and check which one worked, they just observed that yes, this standard v1 cat has some specific hard-wired developmental tendencies. It is not surprising that taking it out of the environment it was evolved for messes things up, but I would not take the extra step that this means that something is essential in general.

Re:John Carmack and Rich Sutton partner to accelerate develo... by funnymony on Saturday October 07, 2023 @02:01PM (#37804018)
This would be like learning everything from books vs learning by experience. (Can’t learn skiing from books. I talk from experience)
Re:John Carmack and Rich Sutton partner to accelerate develo... by bch on Saturday October 07, 2023 @09:21PM (#37807015)
Clever - reminds me of:

In school, we learn the lesson then get the test. In life we get the test then learn the lesson.

Re:John Carmack and Rich Sutton partner to accelerate develo... by og_kalu on Saturday October 07, 2023 @05:08PM (#37805589)
LLMs already do this in their own way. It's not like they learn to predict through magic osmosis. They make predictions and depending on how right or wrong they are, their neurons are shifted by grad descent to accommodate this error. Then they predict again.

The Text is the environment, The Text is the world and they are very much interacting with it.

Re:John Carmack and Rich Sutton partner to accelerate develo... by bwv848 on Sunday October 08, 2023 @03:42AM (#37808603)
Not really, LLM Decoders(without RLHF) are usually trained by teacher forcing, that means they just get learning singals from just predicting one token.
Re:John Carmack and Rich Sutton partner to accelerate develo... by Andrex on Saturday October 07, 2023 @05:58PM (#37805904)
I can see us walking into a situation where we make a robot that has motor functions, but we don't give it the equivalent of nerves and nerve endings so and so it doesn't feel pain. That would definitely warp a human's development.

But then, is it ethical to build in pain receptors? Feels like we're brushing up against the territory of gods.

Re:John Carmack and Rich Sutton partner to accelerate develo... by gls2ro on Sunday October 08, 2023 @04:07AM (#37808697)
What would pain receptors be in a robot?

I ask this as I think the ethical dilemma should more about implementing suffering as a long term damaging effect of a stimuli.

When I think about pain I see it as a level that triggers a need to change something. Our brains are perceiving this level in a very uncomfortable way so we call this pain.

So I really think that without consciousness we cannot implement pain or suffering as we humans perceive it.

Re:John Carmack and Rich Sutton partner to accelerate develo... by Andrex on Thursday October 12, 2023 @08:19PM (#37864879)
> So I really think that without consciousness we cannot implement pain or suffering as we humans perceive it.

Perhaps, but I guess it would depend on your definition of consciousness. We do not extend that definition to most animals and yet they definitely feel pain.

You're right, the "feeling" of pain is ephemeral and hard to translate to a computer program. It would probably take a while doing potentially nasty experiments (to both man and machine) to nail it down.