Re:Ask HN: $8k on AWS in August for you, any interesting com...
by lucb1e on Monday August 03, 2015 @01:38PM
(#9998099)
If I get this correctly, this would help make a backup of archive.org. It would download a big amount of data and then store it for... one month, until the instances expire? That seems rather wasteful of bandwidth and effort, or am I missing the point?
[ Parent
| on HN ]
Re:Ask HN: $8k on AWS in August for you, any interesting com...
by sp332 on Monday August 03, 2015 @01:45PM
(#9998173)
That's a separate AT project. The Warrior downloads data from various websites (usually ones that look like they're about to go down) and uploads the data to a central AT server. It's distributed to get around rate-limiting and banning. Data is not stored long-term on the Warrior.