News for hackers. Stuff that matters.
Re:Robots.txt meant for search engines don’t work well for w... by hyperdunc on Friday April 21, 2017 @01:22PM (#14167444)
Perhaps implicit in the comment you replied to is the idea that there's no such thing as a "content owner".
Re:Robots.txt meant for search engines don’t work well for w... by un-devmox on Friday April 21, 2017 @03:06PM (#14168385)
That's the debate I was trying to initiate!

For instance, is it OK to crawl a blog with explicit copyright, save that data, then publish it elsewhere?