I think there is a lot of potential for transformers in RL. And I do think that AGI is mostly a matter of scale. But I do not agree at all that AGI is just transformers + RL. I don’t think we have a concrete idea at all, and I don’t agree at all that 10 years ago people thought we had no path forward. 10 years ago is exactly AlexNet, aka the birth of deep learning.

I think it's fair to say that no one knows if AGI is even possible at this point, but RL+transformer is definitely the most promising approach imo.

10 years ago no one was talking about AGI, researchers didn't even call it "AI" due to its association with the previous AI winter. There were certainly advances in ML but I think if you asked anyone whether those approaches would lead to real AI it would be an unequivocal no.

I'd say AGI is very very much possible because it already exists (for example: humans) and unless there exists a soul or some other kind of "literally magic" (and I'd be doubtful even then) there is no reason we couldn't one day build AGI.

It also seems like unless something goes very very wrong with the exponential growth curve in computational capacity humanity has had, we'll be able to straight up simulate a human brain (probably well before) 2100.

One big issue is how exactly we'll continue to scale. Exponential growth is hard to maintain.

An example: In the Chinchilla paper [1], the authors suggest that most big transformer models are undertrained, and that we will probably see diminishing returns in scaling up the size of networks if we don't also scale up the size of the datasets. They have a subanalysis where they extrapolate out how big datasets will need to be for larger models. If you believe it, then within two to three generations of big NLG models, we may need on the order of 100 trillion text tokens. But it's not clear if that much text even exists.

[1] https://arxiv.org/abs/2203.15556

The scaling laws of large language models are very specific to language models and the way they're trained. The important thing that LLMs demonstrate is that transformers are capable of this kind of scale (where other approaches have not)

In the RL space, a sufficiently complex, stochastic environment is effectively a data generator.

I'm not sure I agree that there is a distinction between scaling laws and a model being "capable of scale." RNNs are Turing complete, so from that perspective they should in theory they should be sufficient for AGI. But of course they are not because their scaling with regards to network depth and the length of sequences is abysmal. LLMs do scale with depth and sequence length, if their scaling laws with regard to dataset size prevent us from training them adequately, then we are stuck nonetheless.

I haven't heard of any groups who are studying data constrained learning in the context of LLMs, but that will probably change as models get bigger. And at that point, architectures with better scaling laws may be right around the corner, or they may not. That's the pain of trying to project these things into the future.

The scaling laws for LLMs depend heavily on the quality of data. For example, if you add an additional 100gb of data but it only contains the same repeating word, that will hurt the model. If you add 100gb of completely random words, that will also hurt the model. Between these two extremes (low and high entropy), human language has a certain amount natural entropy that helps the model gauge the true co-occurrence frequency of the words in the sentence. The scaling laws for LLMs aren't just a reflection of the model but the conditional entropy of human-generated sentences.

RL is such a different field that you can't apply these scaling laws directly. eg. agents playing tictactoe and checkers would stop scaling at a very low ceiling.

One possible risk I see is that with the amount of model generated text out there it will at some point inevitably result in feeding the output of one model into another unless the source of the text is meticulously traced. (My assumption is that that would hurt the model that you are trying to train as well.)