News for hackers. Stuff that matters.

A comment on List of apps people pay for but have low rating. Parent comment

Re:List of apps people pay for but have low rating by uglygoblin on Saturday May 01, 2021 @05:57PM (#27010262)
Robots.txt can accomplish the same thing which means they periodically grab a single text file from you.
[ Parent | on HN ]
Re:List of apps people pay for but have low rating by account42 on Monday May 03, 2021 @10:07AM (#27025253)
> robots.txt can accomplish the same thing

Not exactly - robots.txt will prevent Google from crawling the page but it might still show up in search results (without a snippet or your page title) based on how others link to it.

[ Parent | on HN ]
Re:List of apps people pay for but have low rating by Google234 on Sunday May 02, 2021 @06:43AM (#27013990)
How dare they!
[ Parent | on HN ]