Re:Ask HN: How do you feel about web scraping?
by alkonaut on Wednesday June 07, 2017 @12:49PM
(#14507889)
I know - but let's assume that since I and google could reach the document it was inadvertently put on a public server. Since it was a mistake it won't be filtered in robots.txt either.
For example, in the weeks before an Apple event, people tend to visit presumed unannounced product pages (by guessing) etc.
For example, in the weeks before an Apple event, people tend to visit presumed unannounced product pages (by guessing) etc.