Think of GPT-3 raw output as a markov chain operating on paragraphs and contexts instead of tokens. You can't write a coherent short story with it without your own "inpainting".

Check out https://novelai.net/ if you're looking for ways to preserve continuity.