Re:How Will AI Learn Next?
by haltist on Friday October 06, 2023 @04:25PM
(#37795894)
In system design there is something called resonant amplification and what you are describing is very similar. The biases of the model are amplified with each iteration and the end result is that the system converges onto the patterns recognizable and amplified by its architecture. If you know about impulse and frequency analysis then an AI system can be considered to be a signal processor that amplifies and attenuates certain frequencies in the input/impulse. Running an LLM in a loop always ends up with nonsense as the final output.
[ Parent
| on HN ]
So if left unchecked, the thing we built in man's attempt to play god could result in gibberish? sounds kind of like the tower of babel; seems humankinds only defense would be creating a new language that the machines can't infiltrate
Another defence is to check them, which is why the thumbs up, thumbs down, and regenerate buttons are there on the ChatGPT user interface.
You can try the experiment yourself. Take any open source LLM and then feed the output back into it in a loop and you'll see what I mean. Most LLMs diverge and lose coherence in less than 100 iterations.