dataslap

21 Gigs in total for everything, ignoring errors (files too big / broken links)

I'm really happy their share it though. The collection is just amazing - it's a good idea for "what books to buy for a personal library when you start making decent money". Definitely would spend money on things like this at their store if they sell them.

I'm at 300 right now, 9.5 gigs. I think some are skipped cause the download times out (can redownload manually I suppose)

P.S. it will take hours or days =))) <3 <3 <3

Please use responsibly. Recommended to delay by a few days - it seems like there is an initial surge going on right now. Increase download delay to 20 sec - 1 minute. Be respectful.

https://github.com/ilyaperepelitsa/met_book_downloader

I think someone else is messing with it - got slower for me too.

It's slow cause things are quite large there (just saw a 2 gig book). I'm using delays, don't worry - hence "in a few hours" <3 <3 <3

<3

Uploading a scrapy crawler that downloads PDF books to github in a few hours, gonna post the link here

some "ethical" measures may do the trick to. scrapy has a setting to integrate delays + you can use fake headers. Some sites are pretty persistent with their cookies (include cookies in requests). It's all case by case basis

depends on the task. For example they have a decent file/image downloading middleware.

scrapy has a pretty decent parser too