Re:Generative Agents: Interactive Simulacra of Human Behavior
by TeMPOraL on Tuesday April 11, 2023 @09:09AM
(#35524601)
> Intelligence/consciousness will emerge, maybe?
Extending that: LLM latent spaces are now some 100 000+ dimensional vector spaces. There's a lot of semantic associations you can pack in there by positioning tokens in such space. At this point, I'm increasingly convinced that, with sufficiently high-dimensional latent space, adjacency search is thinking. I also think GPT-4 is already close to be effectively a thinking entity, and it's more limited by lack of "inner loop" and small context window than by the latent space size.
Also, my kids are ~4 and ~2. At times they both remind me of ChatGPT. In particular, I've recently realized that some of their "failure modes" in thinking/reacting, which I could never describe in a short way, seem to perfectly fit the idea of "too small context window".
Extending that: LLM latent spaces are now some 100 000+ dimensional vector spaces. There's a lot of semantic associations you can pack in there by positioning tokens in such space. At this point, I'm increasingly convinced that, with sufficiently high-dimensional latent space, adjacency search is thinking. I also think GPT-4 is already close to be effectively a thinking entity, and it's more limited by lack of "inner loop" and small context window than by the latent space size.
Also, my kids are ~4 and ~2. At times they both remind me of ChatGPT. In particular, I've recently realized that some of their "failure modes" in thinking/reacting, which I could never describe in a short way, seem to perfectly fit the idea of "too small context window".