Re:DeepSearcher: A Local open-source Deep Research
by parhamn on Tuesday February 25, 2025 @04:30PM
(#43177696)
Is this true? Wouldn't all the "site to markdown" type services be infringing then?
Re:DeepSearcher: A Local open-source Deep Research
by tekacs on Tuesday February 25, 2025 @07:36PM
(#43179288)
Google and Bing's Cache, Archive.org, Archive.is, CommonCrawl... many services have previously or currently presented the full document.
Google and Bing removed their cache features when LLMs started taking off – as I said in a sibling comment, I wonder if they felt that that regime was finally going to be challenged in court as people tried to protect their data.
That being said, "can't present the full document due to copyright" seems at odds with all of the above examples existing for years.
Re:DeepSearcher: A Local open-source Deep Research
by zk108 on Thursday February 27, 2025 @11:10AM
(#43195678)
(founder here) We are working on that problem of providing deeper level of search especially on proprietary datasets (think reference works, books, papers etc.). Started off with Arxive papers )We are working on that problem of providing deeper level of search especially on proprietary/ copyright datasets (think reference works, books, papers etc.). We are working with a number of large publishers on this.
We started off with Arxive papers to test out the product- would love to get feedback :)
I wrote my own implementation using various web search APIs and a puppeteer service to download individual documents as needed. It wasn't that hard but I do get blocked by some sites (reddit for example).