Re:Neeva (search engine with a privacy and AI focus) is shut...
by ragnarsson on Saturday May 20, 2023 @02:22PM
(#36014314)
I have written crawlers for a while, most sites are unfriendly to anything other than Google (maybe they spare bing as well), some make it downright impossible without extensive work on evading their blocking techniques.
I expect (but can not confirm) this is the same reason why search engines like ddg and you.com also rely on bing's indexes.
Google's supremacy is here to stay. Only threat is bing or someone that comes with some excellent tech that bing incorporates (like openai) but actually works for search and is not just a LLM.
This combined with the reports that Bing failed to cut into Google’s market show that Google is safe for a while.
I wonder if I’ll see disruption of Google’s core business in my lifetime. I really want it to happen but it’s a mighty quest