News for hackers. Stuff that matters.
Re:9th Circuit holds that scraping a public website does not... by nojvek on Tuesday September 10, 2019 @02:06PM (#20931245)
you can have your website offlimits from google by having a robots.txt too. What's the problem? People willingly want google to index them so they appear in search results and google can send them traffic.

There are no comments on this comment yet.