News for hackers. Stuff that matters.

A comment on Yahoo is making a return to search. Parent comment

Re:Yahoo is making a return to search by astrange on Monday January 30, 2023 @08:47PM (#34589779)
Your idea for a search competitor is to ignore robots.txt?
[ Parent | on HN ]
Re:Yahoo is making a return to search by VWWHFSfQ on Monday January 30, 2023 @09:03PM (#34589917)
or an advertising competitor that ignores DNT!

oh wait

[ Parent | on HN ]
Re:Yahoo is making a return to search by berkle4455 on Monday January 30, 2023 @10:12PM (#34590386)
It's a just a text file.
[ Parent | on HN ]
Re:Yahoo is making a return to search by chihuahua on Monday January 30, 2023 @10:17PM (#34590428)
This seems perfectly logical to some people:

Google ignores robots.txt: "Google is evil! They're trespassing on my webserver!"

Google follows robots.txt: "Google search results suck! They're not indexing GitHub!"

[ Parent | on HN ]
Re:Yahoo is making a return to search by Nuzzerino on Tuesday January 31, 2023 @05:54AM (#34593644)
Yes, clearly that's the best possible interpretation of what I said. /s
[ Parent | on HN ]