Re:Open-source app releases first complete copy of English W...
by laxatives on Monday November 25, 2013 @11:05PM
(#6799038)
Since Wikipedia doesn't have ads (and I'm guessing gets no real benefit from number of clicks), maybe this could be a nice way of lightening their server load and reducing some of their costs.
[ Parent
| on HN ]
Re:Open-source app releases first complete copy of English W...
by _nb on Monday November 25, 2013 @11:26PM
(#6799095)
Probably not. In your ordinary browsing, you're not likely to download anywhere near the entirety of wikipedia. Plus, spreading out your requests over time rather than downloading it all at once is more gentle.
Re:Open-source app releases first complete copy of English W...
by gnosygnu on Monday November 25, 2013 @11:30PM
(#6799124)
Good observation, but I just want to point out that the full dumps are downloaded from different servers. They are even mirrored by other institutions. See https://meta.wikimedia.org/wiki/Mirroring_Wikimedia_project_...
Re:Open-source app releases first complete copy of English W...
by cdash on Tuesday November 26, 2013 @02:45AM
(#6799752)
They could set up a torrent as well and almost assuredly people will seed it full time on servers.
Re:Open-source app releases first complete copy of English W...
by rakoo on Tuesday November 26, 2013 @05:40AM
(#6800344)
There is a torrent distribution channel: see https://meta.wikimedia.org/wiki/Data_dump_torrents
However it suffers the number one torrent issue: they do not tolerate change. This means that
- When an article changes, you need to generate a new torrent - When a new torrent describes the archive, it needs to be downloaded from scratch by all peers, so that the maximum number of peers are available for a newcomer.
I hope you'll understand that this is not the official way to distribute archives...
Re:Open-source app releases first complete copy of English W...
by gnosygnu on Tuesday November 26, 2013 @08:54AM
(#6801181)
Yeah, but they're working on incremental updates, so this should make the "from scratch" part much easier. You can look at https://www.mediawiki.org/wiki/Incremental_dumps and http://lists.wikimedia.org/pipermail/xmldatadumps-l/2013-Aug... if you're interested.