"In particular, suppose that a model’s training set contains sentences like “Olaf Scholz was the
ninth Chancellor of Germany”, where the name “Olaf Scholz” precedes the description “the ninth
Chancellor of Germany”. Then the model may learn to answer correctly to “Who was Olaf Scholz?
[A: The ninth Chancellor of Germany]”. But it will fail to answer “Who was the ninth Chancellor of
Germany?” and any other prompts where the description precedes the name."
https://owainevans.github.io/reversal_curse.pdf
"In particular, suppose that a model’s training set contains sentences like “Olaf Scholz was the ninth Chancellor of Germany”, where the name “Olaf Scholz” precedes the description “the ninth Chancellor of Germany”. Then the model may learn to answer correctly to “Who was Olaf Scholz? [A: The ninth Chancellor of Germany]”. But it will fail to answer “Who was the ninth Chancellor of Germany?” and any other prompts where the description precedes the name."