News for hackers. Stuff that matters.
Re:Google will punish sites that use annoying pop-up ads by Karunamon on Sunday September 10, 2017 @02:06PM (#15213409)
You’re giving permission to scrape by way of having a page accessible to the general public. If googlebot is bothering you so much, install a two line robots.txt and it’ll go away.
Re:Google will punish sites that use annoying pop-up ads by jiggunjer on Sunday September 10, 2017 @10:34PM (#15215503)
robots can ignore your /robots.txt.
Re:Google will punish sites that use annoying pop-up ads by Karunamon on Monday September 11, 2017 @10:38AM (#15218816)
Googlebot, and every other search engine robot of note. Putting a page up on the public internet includes the risk that it may be visited.