News for hackers. Stuff that matters.
Re:Language models are nearly AGIs but we don't notice by krackers on Sunday November 13, 2022 @06:58PM (#33588671)
>long term memory, short term memory

Doesn't that mean the architecture isn't settled? The current mechanism of appending the output of the model back into the prompt feels like a bit of a hack. I'm only a layman here but it seems transformer models can only propagate information layer-wise, adding some ability to propagate information across time like RNNs do might be useful for achieving longer-term coherence.

There are no comments on this comment yet.