we are strategically prompting it as such: https://news.ycombinator.com/item?id=33750013

open to all ideas here

Is a "prompt prefix" a thing in GPT-3 or do you mean you combine that prefix with the text of the page into a prompt?

We prepend the prompt prefix to the page text to create the actual model prompt. GPT doesn't have any such abstraction.

And GPT-3 remembers the prefix at the end of a long story and writes a summary?

That sounds very surprising to me. I would have strongly expected it continues the story.