News for hackers. Stuff that matters.

A comment on Robots.txt for the NYT has a specific exclusion for an 1996 news article. Parent comment

Re:Robots.txt for the NYT has a specific exclusion for an 19... by Isthatablackgsd on Monday March 29, 2021 @11:10AM (#26622885)
I thought I am the only one! Last week, I was trying to find something with the quotes and it completely ignore it. It took me a few queries to find what I am looking for.
[ Parent | on HN ]
Re:Robots.txt for the NYT has a specific exclusion for an 19... by bbarnett on Wednesday March 31, 2021 @05:21AM (#26645099)
Use verbatim, as per my comment up thread.
[ Parent | on HN ]