Re:Ask HN: Is AGI 'computable' on transistor substrates? Is ...
by mikewarot on Wednesday March 13, 2024 @04:58PM
(#39697434)
I believe that MemGPT makes AGI possible this year, if it hasn't already happened.
Thanks to the structure inherent in writing, Word2Vec maps words into a coherent latent space.
Thanks to positional encoding, self attention mechanisms are enabled. Giving rise to next word prediction at superhuman levels in Transformer LLMs.
Thanks to synthetic data, you can train an LLM to follow directions.
Thanks to being able to train LLM s, you can generate synthetic examples of them successfully using a memory prosthesis.
They can be trained on those examples, and suddenly you have MemGPT which can be a packer[1].
It remains to be seen if a mapper can also be emulated by an LLM. I'm not sure how you could generate a ton of examples of finding an inconsistency between internal chunks of knowledge for it to train against.
Thanks to the structure inherent in writing, Word2Vec maps words into a coherent latent space.
Thanks to positional encoding, self attention mechanisms are enabled. Giving rise to next word prediction at superhuman levels in Transformer LLMs.
Thanks to synthetic data, you can train an LLM to follow directions.
Thanks to being able to train LLM s, you can generate synthetic examples of them successfully using a memory prosthesis.
They can be trained on those examples, and suddenly you have MemGPT which can be a packer[1].
It remains to be seen if a mapper can also be emulated by an LLM. I'm not sure how you could generate a ton of examples of finding an inconsistency between internal chunks of knowledge for it to train against.
[1] https://reciprocality.org/Reciprocality/r0/index.html#:~:tex....