News for hackers. Stuff that matters.

A comment on Robots.txt for the NYT has a specific exclusion for an 1996 news article. Parent comment

Re:Robots.txt for the NYT has a specific exclusion for an 19... by FateOfNations on Tuesday March 30, 2021 @01:58AM (#26631582)
That was mostly PSF's fault since they had the default documentation URLs setup to point at the Python 2 documentation for way longer than they should have...
[ Parent | on HN ]
Re:Robots.txt for the NYT has a specific exclusion for an 19... by disgruntledphd2 on Tuesday March 30, 2021 @06:55AM (#26633127)
Oh really? Interesting. I always thought it was because Google never switched to Python 3.
[ Parent | on HN ]