News for hackers. Stuff that matters.
Re:Every BBC Essential Mix, 1993 to 2010 by brighton36 on Saturday September 04, 2010 @06:12PM (#1663509)
I may regret posting this ....

But this is what you probably want to do:

user@host:/path/to/EssentialMixRip09.04.10$ wget -rl 0 -np -A .mp3 http://thenine.ca/essential/

Re:Every BBC Essential Mix, 1993 to 2010 by _delirium on Sunday September 05, 2010 @05:27AM (#1664270)
As a heads up for anyone who doesn't realize how much music this is, that wget will slurp >100gb of stuff (or would, if it weren't for sever load).
Re:Every BBC Essential Mix, 1993 to 2010 by lsb on Saturday September 04, 2010 @07:25PM (#1663646)
Using Coral CDN:

  wget -rl 0 -np -A .mp3 http://thenine.ca.nyud.net/essential/
Re:Every BBC Essential Mix, 1993 to 2010 by aidenn0 on Sunday September 05, 2010 @12:00AM (#1663968)
Nogood, checkout the robots.txt
Re:Every BBC Essential Mix, 1993 to 2010 by _delirium on Sunday September 05, 2010 @02:43AM (#1664137)
Since Coral considers itself a distributed caching proxy rather than a crawler, it doesn't respect robots.txt. It does respect the relevant cache-control headers, like "no-cache", but this site doesn't appear to set them.

However, the main problem here is that Coral doesn't cache any files >50mb, so almost none of these are cached. The few <50mb do seem to be, though, e.g.: http://thenine.ca.nyud.net/essential/1998/1998.01.01%20-%20E...

Re:Every BBC Essential Mix, 1993 to 2010 by forkqueue on Sunday September 05, 2010 @02:12AM (#1664114)
Yup, thenine.ca is blocking coral cache for .mp3 :(
Re:Every BBC Essential Mix, 1993 to 2010 by jc-denton on Sunday September 05, 2010 @04:05PM (#1665061)
-e robots=off

However aren't they copyrighted?!

Re:Every BBC Essential Mix, 1993 to 2010 by j2d2 on Saturday September 04, 2010 @06:23PM (#1663537)
Or install extension.fm :)