News for hackers. Stuff that matters.
Re:Full-history English Wikipedia dump produced: 5.6TB uncom... by _delirium on Wednesday April 14, 2010 @03:49PM (#1265943)
Yeah, the deleted articles could probably be automatically filtered, at least on a fail-safe basis by only dumping the ones that have a known "not a legal issue" deletion reason, like "notability" (there are even semi-formalized deletion codes, itself a mild absurdity). There are probably non-technical / non-legal reasons people don't want to dump them, but there's also some of just a "not a priority" aspect. The dump reported in this story is actually the first successful full-history dump in quite some time, because the dump scripts were perennially broken / bogging down due to the size of the data / crashing due to MySQL weirdness. So most of the dump effort has been on just getting the official stuff out. Next up on the priority list will probably be some way of doing image dumps.

You do get a bit of a warning if you recreate a deleted page. When you go to the editing screen at the title of an article that was previously deleted, it'll show you the summary from the deletion log at the top, and ask you if you're sure you want to recreate it. There's also a "nothing can go here" protected placeholder used for articles that are persistently being recreated, which'll make it impossible to edit at that location.

Yeah, I can sympathize on the Wikipedia-inner-sanctum thing. I was actually pretty deeply into it (I've been an admin since '04, was formerly on the Arbitration Committee, formerly active on the mailing lists, etc.), but as the Policy And Process kept accumulating, I lost interest in navigating it, so am more on the periphery these days. It's probably inevitable that things would go that direction, because in the early days there were probably <100 Wikipedians active enough to form the Wikipedia Cabal, all of whom at least recognized each others' names, so stuff could be pretty informal. But it's hard to scale that up to a site with 1700 admins and 15k+ editors. A lot of things are kind of lame about how things are organized these days, but honestly I have no idea how I'd do it better; despite its flaws it's often still amazing to me that Wikipedia works at all.

There are no comments on this comment yet.