Re:Data-Mining Wikipedia for Fun and Profit
by learc83 on Thursday August 19, 2021 @07:35PM
(#28240634)
I don’t think the op was talking specifically to the content author, but to all the people who read the article and get the idea to scrape Wikipedia.
[ Parent
| on HN ]
Re:Data-Mining Wikipedia for Fun and Profit
by bawolff on Thursday August 19, 2021 @07:43PM
(#28240703)
Honestly i'd rather people err on the side of scrapping wikipedia too much than live in fear of being disruptive and not do cool things as a result. Wikipedia is meant to be used to spread knowledge. That includes data mining projects such as the one in this blog.
(Before anyone takes this out of context - no im not saying its ok to be intentionally disruptive, or do things without exercising any care at all. Also always set a unique descriptive user-agent with an email address if you're doing anything automated on wikipedia).
Re:Data-Mining Wikipedia for Fun and Profit
by learc83 on Friday August 20, 2021 @08:24PM
(#28252895)
Having been on the other side of this, I’d rather we encourage people to make use of formats/interfaces designed for machines and use the right tool for the job instead of scraping everything.
It’s incredibly easy for careless scrapers to disrupt a site and cost real money without having a clue what they’re doing.
I want people to think twice and consider what they are doing before they scrape a site.