News for hackers. Stuff that matters.

A comment on 9th Circuit holds that scraping a public website does not violate the CFAA [pdf]. Parent comment

Re:9th Circuit holds that scraping a public website does not... by jjohansson on Tuesday September 10, 2019 @12:00AM (#20924933)
They still scrape and index sites blocked by robots.txt, but they often don’t display those sites in their SERPs (but sometimes they still do)
[ Parent | on HN ]
Re:9th Circuit holds that scraping a public website does not... by C14L on Tuesday September 10, 2019 @06:44AM (#20926795)
Never seen it in any logs in 20 years.

Do you have a source for that claim?

[ Parent | on HN ]
Re:9th Circuit holds that scraping a public website does not... by jjohansson on Wednesday September 11, 2019 @12:23PM (#20941878)
Looks like I’m wrong. They will rank pages blocked by robots.txt, but won’t index them.

https://www.google.ca/amp/s/www.searchenginejournal.com/goog...

[ Parent | on HN ]