News for hackers. Stuff that matters.

A comment on Qwant: The Search Engine That Respects Your Privacy. Parent comment

Re:Qwant: The Search Engine That Respects Your Privacy by arminiusreturns on Friday November 13, 2020 @02:30AM (#25079923)
Why are so many search engines trying to piggy back off other search engines? Can they not crawl sites themselves and cut out the extra layer? In a sea of bad websites I feel like if you started with a whitelist approach it could be a good basis.
[ Parent | on HN ]
Re:Qwant: The Search Engine That Respects Your Privacy by hosteur on Friday November 13, 2020 @02:56AM (#25080080)
Crawling is difficult these days as recaptcha and other measures keep bots out if you are not google or MS
[ Parent | on HN ]
Re:Qwant: The Search Engine That Respects Your Privacy by NetOpWibby on Friday November 13, 2020 @10:55AM (#25083525)
That is quite unfortunate. How the hell are new original search engines supposed to compete?
[ Parent | on HN ]
Re:Qwant: The Search Engine That Respects Your Privacy by jameshush on Friday November 13, 2020 @01:20PM (#25085546)
I’ve worked in this space for over two years. They can’t. It’s simply not possible. The only feature that’s feasible to offer is privacy. You’ll never be able to catch up to Google or Bing with crawling.
[ Parent | on HN ]
Re:Qwant: The Search Engine That Respects Your Privacy by NetOpWibby on Friday November 13, 2020 @01:42PM (#25085816)
In your opinion, do you think a new original search engine can survive in this landscape? Of course it would always be behind Google and Bing by about 15 years.

I've been reseasrching some ideas.

[ Parent | on HN ]
Re:Qwant: The Search Engine That Respects Your Privacy by crocodiletears on Friday November 13, 2020 @10:55AM (#25083532)
It's a pretty herculean task, moreso no than ever before. On top of the sheer volume of the internet, and the technical challenges of performant search, many sites are hostile to unknown scrapers, and automation.
[ Parent | on HN ]