Are there any large publicly available archives of usenet one can download?

I need it for an information retrieval research project. I am aware of gmane.org, but do not think they allow bulk download.

And https://archive.org/details/usenethistorical and https://archive.org/details/giganews

Please use the bittorrent download option if possible. It reduces load on archive.org.

If there are any seeds, sure. Torrents are most useful for new, big, popular items.

IA torrent files use the archive as web seeds if they have to. But if there's a spike in interest - like right now apparently - it would reduce the load. So it will still work if there are no seeds, and it will reduce load on the servers when that's possible.

Edit: this is all just to be polite, since the archive is not worried about using a ton of bandwidth.

Is the code that runs the frontend of the IA open source? I'd be interested in contributing to that, so when requests for certain objects are creating excessive load, a response status code is provided to indicate so, and the alternate URI returned is a magnet link for the object.

EDIT: It appears an HTTP 303 status code accomplishes this

503, we return the standard 503 code. Remember that most of our users don't know what BitTorrent is, and would prefer that the archive Just Worked.

Right! I'm not interested in breaking the Internet Archive, and I'd expect it to move to IPFS [1] eventually (content addressable web) [2]. If/when/how that happens, I'd expect traditional http tooling to still work (curl, wget, etc), which is why I went looking for a status code that indicates an alternate path for the resource/content.

That's why my above comment kept getting edited as I did some more research. 503 is an ugly failure. 429 tells the client to back off, but it doesn't provide a fallback to still get the content. 303 does.

I thought this train of thought was in line with Brewster's blog post [2]. Apologies for the confusion!

[1] https://ipfs.io/

[2] http://brewster.kahle.org/2015/08/11/locking-the-web-open-a-...

From the x86-64 archive:

> Actually, I'm a bit disgusted at Intel for not even _mentioning_ AMD in their documentation or their releases, so I'd almost be inclined to rename the thing as "AMD64" just to give credit where credit is due.

TIL.

I remember the IA-16, IA-8, and IA-4 articles on Wikipedia:

https://en.wikipedia.org/w/index.php?title=IA-16&oldid=13741...

https://en.wikipedia.org/w/index.php?title=IA-8&oldid=137416...

https://en.wikipedia.org/w/index.php?title=IA-4&oldid=195098...

YArchive is one of my favorite high-signal low-noise sites; so much that I run a mirror of it.

Glad Tomte shared it as the Computers archive already had some interesting nuggets of wisdom:

http://yarchive.net/comp/index.html

Included were arguments against triple modular redundancy in favor of other models, the uses of zero registers + proposal to use them on floating-point, and Mashey's algorithm counterpoint to Knuth's work. Those are a few I found quickly that were good and non-obvious to most at some point. There's a lot of stuff on here but the links are each short for easy skimming.

Good resource.

Some of my favorites are:

http://yarchive.net/comp/linux/struct_declarations.html

http://yarchive.net/comp/linux/strncpy.html

http://yarchive.net/comp/linux/gpl.html

http://yarchive.net/comp/linux/export_symbol_gpl.html

http://yarchive.net/physics/anthropic_principle.html

http://yarchive.net/comp/compiler.html

http://yarchive.net/comp/aliasing.html

http://yarchive.net/comp/linux/typedefs.html

http://yarchive.net/comp/const.html

http://yarchive.net/comp/linux/dev_random.html

The space category is fascinating, too.

Those were some good reads. Appreciate it. The use of API naming to indicate intent for a license was particularly clever.

is usenet still used for discussion? how is it?

i know it's used for binaries, are there any frequent usenet users around that don't use it for binaries?

I lurk and occasionally post on a dozen or so groups.

Where's the rest of the comp. domain?

The archives are not complete - it's just articles of interest to one particular person. See http://yarchive.net/about.html