News for hackers. Stuff that matters.
Re:Brave Search replaces Google as default search engine in ... by Closi on Wednesday October 20, 2021 @07:19AM (#28929352)
I would assume this is easier if you are indexing more frequently and have been indexing for longer, and Google has more resources to allow them to crawl more often (another advantage to scale in search, which combined with other advantages of scale, is why Google has an effective monopoly IMO).

If you recently started crawling you won't be able to detect which content was 'first', and if you don't crawl as frequently as google you are reliant on your crawl frequency being quicker than the 'content stealing' speed.

Re:Brave Search replaces Google as default search engine in ... by addingnumbers on Wednesday October 20, 2021 @12:13PM (#28932469)
> if you don't crawl as frequently as google you are reliant on your crawl frequency being quicker than the 'content stealing' speed.

No problem there. If a page hasn't been crawled, it won't be a search result.

Re:Brave Search replaces Google as default search engine in ... by Closi on Wednesday October 20, 2021 @05:54PM (#28937274)
I mean you need to crawl often enough that you can determine which site stole content.

If a full crawl of the web takes 3 months and A steals content from B after 2 weeks, it may not be possible to easily infer that A has stolen from B rather than visa-versa.

If I crawl every week and see that A posted it on the first crawl, and B suddenly also had that content on the second crawl, I can infer that B stole from A.