Re:We can't have nice things because of AI scrapers
by tonyhart7 on Tuesday January 13, 2026 @08:17PM
(#46611095)
I mean this AI data scrapper would need to scan and fetch billions of website
why would they even care over 1 single website ??? You expect instiution to care out of billions website they must scrape daily
Re:We can't have nice things because of AI scrapers
by what on Tuesday January 13, 2026 @08:39PM
(#46611257)
This is probably the reason. It’s more effort to special case every site that offers dumps than to just unleash your generic scraper on it.
For this strategy to work, people need to actually use the DB dumps instead of just defaulting to scraping. Unfortunately scraping is trivially easy, particularly now that AI code assistants can write a working scraper in ~5-10 minutes.