News for hackers. Stuff that matters.
Re:Scrape like the big boys by matheusmoreira on Friday November 05, 2021 @11:58AM (#29121095)
This is the number one reason to scrape websites. It's always nice when there's an API with documentation and rate limiting rules you can follow. Sometimes the data I need just isn't there, though. Then I open up their site and find a huge amount of private API endpoints that do exactly what I want. Then I open up a ticket about it and it gets 200 replies but they ignore it for years. It's fucking stupid and it's really no wonder people scrape their site.

There are no comments on this comment yet.