News for hackers. Stuff that matters.
Re:Robots.txt for the NYT has a specific exclusion for an 19... by notacoward on Monday March 29, 2021 @09:41AM (#26621376)
Similar thing here. I wrote a blog post about an online "contest" that was actually a bit of a scam. For years afterward I'd periodically get email from the person asking to take the blog post down because its prominence in search results was souring job prospects. I eventually relented, not because I cared about him but because he mentioned that he had started a family and it was affecting them as well. I didn't want to hurt innocent people. I didn't want to censor my blog either, but I did add a robots.txt so it wouldn't show up in search results.

Since then it seems he really has gotten it together, and even had a project show up on the first page here. So now I suppose the robots.txt entry doesn't matter much, but it's still there anyway.

There are no comments on this comment yet.