The only thing here that current language models objectively don't have is a persistent memory. GPT-3 already uses 2048 tokens of context as input, but it's not persistent. An improved language model would likely need to have persistent memory. There's no obvious reason why the rest of that stuff can't occur as emergent behaviours.

It was a rhetorical question: there is no possible distinction between a highly sophisticated language model and an AGI. If a language model can't produce all the same answers that an AGI can produce, then it just hasn't reached the level of sophistication necessary to do that.

> It was a rhetorical question: there is no possible distinction between a highly sophisticated language model and an AGI. If a language model can't produce all the same answers that an AGI can produce, then it just hasn't reached the level of sophistication necessary to do that.

This is a weirdly dogmatic position.

I would think that an AGI can start a conversation. Can a language model?

Software can't execute itself. It must be executed by an external agent and given some kind of input. That will be true regardless of whether you consider the software to be a true AGI or a mere language model. So what does it mean exactly to "start a conversation"? Are you saying that it should be able to give a useful output with a blank input? I don't see why a language model can't do that.

In principle it might be possible for a language model to develop true general intelligence. However there's no mechanism in GTP and similar systems that makes such a development necessary.

In fact it seems likely that the development of internal structures that might develop into the faculties needed by an AGI would, in their intermediate state, make it worse as a language model. Evolution sometimes faces such development gaps, where to develop a capability that would ultimately grant improvements, it would have to go through intermediate phases that would make it worse. Language models are optimised in a very specific way and trained to solve a very narrow problem, compared to the problems faced by physical beings.

So while I cant honestly say no, a language model can never do that, equally there's no actual reason to believe one ever would.

Was there a mechanism in the evolutionary system of biological life on Earth that makes the development of higher intelligence necessary? How did we develop higher intelligence if it's not possible to get there though a series of progressive improvements?

Furthermore I would argue that intelligence is a necessary development in order to create the most sophisticated possible language model, for the reasons described above. Anything less will not be able to perfectly emulate the conversations of other intelligent beings.

I agree the best possible language model would possess general AI, but you can't go from no intelligence to general AI in one step. It seems likely to me that the intermediate levels of intelligence such a system might develop early on might actually make it a worse language model. The evolutionary path from dumb language model to language model with human level AI might well lie through an unhappy valley in which the fledgling AI makes more mistakes and is easier to catch out than a pure language model. In which case, a system would never take that evolutionary path.

This is a problem with systems optimising towards a single specific function. It constrains their optimisation paths.