Re:Fastly Outage
by ornornor on Tuesday June 08, 2021 @07:59AM
(#27433945)
Try “site:old.Reddit.com”
[ Parent
| on HN ]
That's not going to work.
$ curl https://old.reddit.com/robots.txt
User-Agent: *
Disallow: /
Also, even if search engines are allowed, old.reddit.com pages are not canonical (<link rel="canonical"> points to the www.reddit.com version, which is actually reasonable behavior), so pages there would not be crawled as often or at all.