Re:Scaling Transformer to 1M tokens and beyond with RMT
by mdp2021 on Monday April 24, 2023 @05:37AM
(#35685442)
That is very interesting.
But - note - the label of "stochastic parrots" is not really relevant to the acquisition of notions and the appearance of some consistency checks, but more to the idea that the ideal system performs the checks regularly and deeply - especially in front of new ideas, of which the ones it outputs are especially relevant. Whatever you utter, you have supposed to have thought about critically with some depth - structurally.
In fact, it is a gamechanger to see that "critical thinking" is possible with LLMs - but do not forget that we have seen many examples in months of that not actually happening.
But - note - the label of "stochastic parrots" is not really relevant to the acquisition of notions and the appearance of some consistency checks, but more to the idea that the ideal system performs the checks regularly and deeply - especially in front of new ideas, of which the ones it outputs are especially relevant. Whatever you utter, you have supposed to have thought about critically with some depth - structurally.
In fact, it is a gamechanger to see that "critical thinking" is possible with LLMs - but do not forget that we have seen many examples in months of that not actually happening.