this is a technique called text tiling which separates a document into semantic chunks

https://github.com/Ighina/DeepTiling

https://medium.com/@ganymedenil/how-to-segment-large-texts-f...