News for hackers. Stuff that matters.
Re:Show HN: R2R – Open-source framework for production-grade... by ocolegro on Tuesday February 27, 2024 @01:10PM (#39527358)
Yes, this is on the shortlist.

Do you have any preferred frameworks?

Re:Show HN: R2R – Open-source framework for production-grade... by gardnr on Wednesday February 28, 2024 @03:54AM (#39535613)
I haven't found any frameworks that offer it. The best explanation of an implementation that can take a stream of unformatted text and map over it to determine when a topic changes is explained in this video: https://youtu.be/8OJC21T2SL4?t=1932

They compute embeddings using a window of three sentences and then compute distance to find the largest deltas to break up the text into "topics". It is computationally expensive.

Re:Show HN: R2R – Open-source framework for production-grade... by gardnr on Wednesday February 28, 2024 @09:01AM (#39538013)
I just noticed this has been added to langchain: https://python.langchain.com/docs/modules/data_connection/do...
Re:Show HN: R2R – Open-source framework for production-grade... by chiccomagnus on Thursday February 29, 2024 @06:01AM (#39548032)
check out this https://preprocess.co