News for hackers. Stuff that matters.
Re:Experimenting with LLM-Based Chunk Enhancement for Better... by danmeier on Friday December 01, 2023 @12:16PM (#38489276)
Why are you not just using the chunkers from unstructured ?
Re:Experimenting with LLM-Based Chunk Enhancement for Better... by dgarnitz on Friday December 01, 2023 @12:28PM (#38489421)
You can use the unstructured chunkers and still use this to enhance the quality of the actual chunk. The chunkers from unstructured help you take a more informed approach for a specific type of data but they don't help you more accurate results for your use case.