Funny how they don't mention the solution: to have AIs actively participate in the real world. I mean, it's hardly what people here want to see happen, but that will solve it.

Current LLMs cannot "actively participate in the real world" as humans do because they cannot actively learn from their interaction with the real world. Their weights are fixed. Their further training and finetuning is mediated by external mechanisms. It has nothing to do with what "people here" want.

Sure, though it seems there are a number of near term paths for improvement. Short and long term memory mechanisms would go a long way towards active learning. Fine tuning could be iteratively performed through mechanisms like low rank adaptation. It's been hypothesized that this is one of the purposes of sleep, and humans are consolidating memories during dreams.

Not true. There's plenty of easy ways to update the weights if you want to do that.

You could for example DPO based on another LLM's evalution of how a conversation is going, or you could use whether the conversation keeps going.

>>>"You could for example DPO based on another LLM's evalution of how a conversation is going,"

You can update the weights based on a single point of data (response was bad) but you probably can't usefully update a model that way.

I can't seem to find anyone actually usefully trying this. Do you know of any data?

No but Google (for example) recommends fine-tuning based on at least 500 examples of question response pairs. DPO as far as I know requires a good and bad example. It's not a technique based on just saying "ai response is bad".

How?

By releasing them from being confined to their own farts.

That frames it like ML-models are caged intelligent beings which would prosper if only someone would set them free...

You know, actual "intelligent beings" are also next-token-predictors that are caged in a mechanism that forces them to give certain outputs ...

I've been coming back to this comment and I'm not sure what you mean exactly.

The brain is trapped in the body, and forced to meet its needs (e.g. regulate hormone levels).

The brain and body are trapped in civilization, and forced to produce "outputs" it considers acceptable.

All three, in turn, are trapped in natural selection.

But that's practically what's being done in supervised learning.

If you would isolate a child in front of a CCTV screen, it also wouldn't magically learn from the images what's our definition of a tree, a bush, a bike, a motorcycle, a car. Someone would have to take it aside and explain it first.

In AI you just don't take it aside, you build a new child.

Supervised learning is "Hello lil CCTV model. Here's a bucket of some 'world' data, labeled in a way for you to ingest. Here, some images of T-R-E-E-S...".

All areas of systems theory [0] whether in the cybernetics [1] of Norbert Wiener [2] or in complex real systems like those Meadows [3] and Forrester [4] studied, the divergence from equilibrium that occurs in positive feedback loops is both well studied, yet strangely hard to understand.

We don't know quite what is going happen, because selective amplification of different components will lead to different effects. It's unpredictable. Sometimes we get oscillation. Sometimes we get chaos or noisy behaviour. This relates to "chaos theory" [5].

sometimes we get a discontinuity, either an instant collapse or a "pole" that shoots off to infinity. Sometimes we get all these things in some strange sequence, although the latter two are usually unrecoverable.

The biological system I think is closest to what we're doing with AI is BSE (Bovine spongiform encephalopathy) [6] when we fed dead cow bones back to living farm animals which selected for prions.

[0] https://en.wikipedia.org/wiki/Systems_theory

[1] https://en.wikipedia.org/wiki/Cybernetics

[2] https://en.wikipedia.org/wiki/Norbert_Wiener

[3] https://en.wikipedia.org/wiki/Donella_Meadows

[4] https://en.wikipedia.org/wiki/Jay_Wright_Forrester

[5] https://en.wikipedia.org/wiki/Chaos_theory

[6] https://en.wikipedia.org/wiki/Bovine_spongiform_encephalopat...