Will be sharing the code next week. The basic idea is to find the embeddings of sentences and then finding the distance in the latent space to see if there is too big a jump in context.

A single sentence can have a lot of different meanings. I wonder if it would be more fruitful to use rolling pairs and triplets? Ie was thinking about how you use two moving averages with different window sizes to detect trend shifts.

Yes. We are using rolling sentences. Not just pairs. But the whole story. I have written a little about this in my previous post, shape of stories, https://gpt3experiments.substack.com/p/the-shape-of-stories-...

So we can see how as we are appending sentences and getting the embedding we get the movement of the story in the latent space.