Re:Web scraping is legal, US appeals court reaffirms
by borski on Monday April 18, 2022 @05:36PM
(#31076913)
I agree it’s not perfect. It’s also significantly better than nothing.
Re:Web scraping is legal, US appeals court reaffirms
by Gigachad on Monday April 18, 2022 @09:23PM
(#31078824)
It's also completely insufficient despite being better. There are so many scraper services that its just a matter of paying a small amount of money to make use of tens of thousands of IPs spread across ISPs and countries.
Re:Web scraping is legal, US appeals court reaffirms
by inshadows on Monday April 18, 2022 @08:28PM
(#31078458)
Can you share where did you get such a nice proxy for 500 proxies? TIA
Re:Web scraping is legal, US appeals court reaffirms
by l33t2328 on Monday April 18, 2022 @04:42PM
(#31076237)
Using proxies to hide your identity to get around a denial of access seems to get awfully close to violating the Computer Fraud and Abuse Act(in USA, at least).
I’m surprised your school was okay with it.
In school I needed to scrape a few hundred thousand pages of a proteomics database website. For some reason you had to view each entry one at a time. There was IP throttling which banned you if you made requests too quickly. But slowing the script to 1 request per second would have taken days to scrape the site. So I paid <$5 for a list of 500 proxy servers and distributed it, completing the task in under half an hour.