News for hackers. Stuff that matters.
Re:Data-Mining Wikipedia for Fun and Profit by matkoniecz on Thursday August 19, 2021 @11:36AM (#28235190)
Scrapping, especially on a large scale, can put a noticeable strain on servers.

Bulk downloads (database dumps) are much cheaper to serve for someone crawling millions of pages.

It gets even more significant if generation of reply is resource intensive (not sure is Wikipedia qualifying for that but complex templates may cause this).

There are no comments on this comment yet.