News for hackers. Stuff that matters.

A comment on Web scraping is legal, US appeals court reaffirms. Parent comment

Re:Web scraping is legal, US appeals court reaffirms by bequanna on Monday April 18, 2022 @04:23PM (#31075976)
As someone that has been on the other end, I can tell you devs don’t want to use selenium or inspect requests to reverse engineer your UI and wish there were more clean APIs.

Have you tried making your UI more challenging to scrape and adding a simple API that requires free registration?

[ Parent | on HN ]
Re:Web scraping is legal, US appeals court reaffirms by JimBlackwood on Tuesday April 19, 2022 @07:17AM (#31081685)
So much this!

I work in E-Commerce and (needless to say) we scrape a lot of websites. Due to our growth and the increase in scrapers we require, I’ve been writing a proposal to a higher up to talk to our biggest competitors to all set up a public API that batches the data for a smaller amount of requests.

It would save everyone quite some traffic and effort.

[ Parent | on HN ]