News for hackers. Stuff that matters.
Re:Full-history English Wikipedia dump produced: 5.6TB uncom... by philwelch on Wednesday April 14, 2010 @11:50AM (#1265212)
Doesn't include deleted articles, so no hope if you want to recover one of them. This is a pity since Wikipedia deletes too many articles.
Re:Full-history English Wikipedia dump produced: 5.6TB uncom... by _delirium on Wednesday April 14, 2010 @01:14PM (#1265423)
That's talked about on and off, but one issue is that they'd have to filter deleted articles by deletion reason, at least broadly into "deleted for legal reasons" and "deleted for non-legal reasons" bins. There'd be no problem distributing a dump of articles deleted due to non-notability, but a dump of articles axed for copyright violation, libel, or other legal issues would be a problem.

For specific articles deleted for non-legal reasons (most commonly notability), you can get a copy from a WP admin. Some have volunteered themselves as willing to answer requests: http://en.wikipedia.org/wiki/Category:Wikipedia_administrato...

Re:Full-history English Wikipedia dump produced: 5.6TB uncom... by pbhjpbhj on Wednesday April 14, 2010 @03:30PM (#1265878)
Surely articles have posted on them, or their talk pages, the reason for deletion to allow for responses? Also there should be some way for people to find that an article has been deleted so that they don't recreate the article and repeat the error. Indeed rather than delete couldn't a placeholder be implemented.

Wrong forum for these suggestions but I've never had the time or inclination to attempt to reach the Wikipedia inner sanctum.

Re:Full-history English Wikipedia dump produced: 5.6TB uncom... by _delirium on Wednesday April 14, 2010 @03:49PM (#1265943)
Yeah, the deleted articles could probably be automatically filtered, at least on a fail-safe basis by only dumping the ones that have a known "not a legal issue" deletion reason, like "notability" (there are even semi-formalized deletion codes, itself a mild absurdity). There are probably non-technical / non-legal reasons people don't want to dump them, but there's also some of just a "not a priority" aspect. The dump reported in this story is actually the first successful full-history dump in quite some time, because the dump scripts were perennially broken / bogging down due to the size of the data / crashing due to MySQL weirdness. So most of the dump effort has been on just getting the official stuff out. Next up on the priority list will probably be some way of doing image dumps.

You do get a bit of a warning if you recreate a deleted page. When you go to the editing screen at the title of an article that was previously deleted, it'll show you the summary from the deletion log at the top, and ask you if you're sure you want to recreate it. There's also a "nothing can go here" protected placeholder used for articles that are persistently being recreated, which'll make it impossible to edit at that location.

Yeah, I can sympathize on the Wikipedia-inner-sanctum thing. I was actually pretty deeply into it (I've been an admin since '04, was formerly on the Arbitration Committee, formerly active on the mailing lists, etc.), but as the Policy And Process kept accumulating, I lost interest in navigating it, so am more on the periphery these days. It's probably inevitable that things would go that direction, because in the early days there were probably <100 Wikipedians active enough to form the Wikipedia Cabal, all of whom at least recognized each others' names, so stuff could be pretty informal. But it's hard to scale that up to a site with 1700 admins and 15k+ editors. A lot of things are kind of lame about how things are organized these days, but honestly I have no idea how I'd do it better; despite its flaws it's often still amazing to me that Wikipedia works at all.

Re:Full-history English Wikipedia dump produced: 5.6TB uncom... by cmelbye on Wednesday April 14, 2010 @02:20PM (#1265654)
Deleted articles are stored in the "archive" table. They may have changed this since then, but last time I checked toolserver (http://toolserver.org/) users had access to that table.