News for hackers. Stuff that matters.
Re:We can't have nice things because of AI scrapers by nostrademons on Tuesday January 13, 2026 @05:40PM (#46609404)
That's sorta what MetaBrainz did - they offer their whole DB as a single tarball dump, much like what Wikipedia does. I downloaded it in the order of an hour; if I need a MusicBrainz lookup, I just do a local query.

For this strategy to work, people need to actually use the DB dumps instead of just defaulting to scraping. Unfortunately scraping is trivially easy, particularly now that AI code assistants can write a working scraper in ~5-10 minutes.

Re:We can't have nice things because of AI scrapers by tonyhart7 on Tuesday January 13, 2026 @08:17PM (#46611095)
I mean this AI data scrapper would need to scan and fetch billions of website

why would they even care over 1 single website ??? You expect instiution to care out of billions website they must scrape daily

Re:We can't have nice things because of AI scrapers by what on Tuesday January 13, 2026 @08:39PM (#46611257)
This is probably the reason. It’s more effort to special case every site that offers dumps than to just unleash your generic scraper on it.
Re:We can't have nice things because of AI scrapers by 8note on Tuesday January 13, 2026 @09:55PM (#46611722)
the obvious thing would be to take down their website and only have the DB dump.

if thats the useful thing, it doesnt need the wrapper

Re:We can't have nice things because of AI scrapers by mr_monkey on Friday January 16, 2026 @06:46AM (#46645493)
Ah, yes, if the mafia comes to threaten your business for a protection racket, just abandon your shop, simple !