News for hackers. Stuff that matters.
Re:Don't sell space in your homelab (2023) by gruez on Sunday April 13, 2025 @05:47PM (#43676102)
>Providers like Oxylabs can be quite restrictive, preventing access to many of the common sites that scrapers choose to target.

Most of them seem pretty reasonable?

"Entertainment & streaming" - who's trying to scrape netflix's library?

"Banking and other financial institutions" / "Government websites" / "Mailing" - seems far more likely it'll be used for credential stuffing than for "scraping".

"Ticketing" - seems far more likely that it'll get used by scalpers than for scraping

The main targets of scraping - e-commerce sites (for price comparisons) and social media networks (for user generated content) are fine to scrape. Is there some use case I'm missing here? Is there a huge contingent of people wanting to scrape ticketmaster or bank of america?

Re:Don't sell space in your homelab (2023) by bdcravens on Sunday April 13, 2025 @11:04PM (#43677707)
I used the term "scrapers" pretty loosely, but yes, in many cases they are more bad actors than actual scrapers. However as they say the list may include other sites, I suspect Oxylab adds sites to the list at the site owners' requests (Amazon, Target, etc are likely to be on those lists)