Re:All of ChatGPT's System Prompts
by dleeftink on Sunday October 15, 2023 @01:26AM
(#37887112)
Similar to cache 'hits or misses', I always thought the idea of the underlying 'knowledge cache' being exhausted (i.e. its embedding space) would fit the bill nicely.
Another way of framing it would be along the lines of 'catastrophic backtracking' but attenuated: a transformer attention head veering off the beaten path due to query/parameter mismatches.
These are by no means exhaustive or complete, but I would suggest knowledge exhaustion, stochastic backtracking, or simply perplexion.
Verbiage along the lines of misconstrue, fabricate and confabulate have anecdotally been used to describe this state of perplexity.
Another way of framing it would be along the lines of 'catastrophic backtracking' but attenuated: a transformer attention head veering off the beaten path due to query/parameter mismatches.
These are by no means exhaustive or complete, but I would suggest knowledge exhaustion, stochastic backtracking, or simply perplexion.
Verbiage along the lines of misconstrue, fabricate and confabulate have anecdotally been used to describe this state of perplexity.