News for hackers. Stuff that matters.
Re:Google’s robots.txt parser is now open source by ahazred8ta on Monday July 01, 2019 @03:44PM (#20329005)
Send 410 Gone with a noindex meta tag in html and X-Robots-Tag?

https://www.searchenginejournal.com/google-404-status/254429 "How Google Handles 404/410 Status Codes" -- "If we see a 410, they immediately convert that 410 into an error rather than protecting it for 24 hours"

There are no comments on this comment yet.