News for hackers. Stuff that matters.

A comment on Robots.txt for the NYT has a specific exclusion for an 1996 news article. Parent comment

Re:Robots.txt for the NYT has a specific exclusion for an 19... by cafard on Monday March 29, 2021 @09:51AM (#26621521)
America collectively has a pretty short attention span.
[ Parent | on HN ]
Re:Robots.txt for the NYT has a specific exclusion for an 19... by orangeoxidation on Tuesday March 30, 2021 @05:49AM (#26632746)
That would be fair for radio or such.

In the internet age it can heavily damage or ruin things like your job and dating prospects permanently.

[ Parent | on HN ]