News for hackers. Stuff that matters.
Re:Wayback Machine director outlines the scale of everyone's... by GordonS on Monday October 08, 2018 @01:12PM (#18169223)
Don't forget that they will need geo-redundancy built in to prevent data loss, and they likely have some kind of offline archive too.

Then there is the running cost to power all those drives.

Also, when you have that many drives, you're going to be constantly replacing dead ones.

I have a feeling this is an expensive operation!

Re:Wayback Machine director outlines the scale of everyone's... by Dylan16807 on Monday October 08, 2018 @03:14PM (#18170388)
> Don't forget that they will need geo-redundancy built in to prevent data loss

This is already factoring in 2x storage, so you just put each half in a different location.

> and they likely have some kind of offline archive too.

Okay so 2 million.

> Then there is the running cost to power all those drives.

Yes, that's one of many things that adds on top of the raw drive cost.

> Also, when you have that many drives, you're going to be constantly replacing dead ones.

It's not actually that bad! If you cram ten thousand drives into a few racks, your drive failures per week will be in the single digits. You could have someone come in once a month.

> I have a feeling this is an expensive operation!

It is, but my core point is that it would be expensive even if the drives were free. You need a certain amount of server horsepower and the physical resources to support it. This implies that you're not getting to stick 60 drives in a single chassis, so you need more labor as well.

There was an estimate last year of $1500 for IA to store and serve a terabyte of data indefinitely. I'd love to see a breakdown of that number, how much goes to day 1 hardware, how much goes to year 1 hosting, how long that hardware is expected to last. But even the day 1 cost is definitely a lot more than $50 for 2TB worth of hard drive.

Re:Wayback Machine director outlines the scale of everyone's... by GordonS on Monday October 08, 2018 @04:06PM (#18170853)
> This is already factoring in 2x storage, so you just put each half in a different location.

My bad, I missed that!

> It is, but my core point is that it would be expensive even if the drives were free.

On reflection, and after reading your reply, I agree :)

> There was an estimate last year of $1500 for IA to store and serve a terabyte of data indefinitely.

I assume that's $1500 per year?

Re:Wayback Machine director outlines the scale of everyone's... by Dylan16807 on Monday October 08, 2018 @05:25PM (#18171456)
> I assume that's $1500 per year?

Total, not per year. Even S3 would only cost $250 a year to store a TB redundantly, and you could host 20 of them off a $200-per-month server or something.

Backblaze gets you a bit more than one copy for only $60 a year.

Re:Wayback Machine director outlines the scale of everyone's... by GordonS on Tuesday October 09, 2018 @04:00PM (#18179347)
I'm not quite understanding how storing a quantity of data indefinitely doesn't have an ongoing cost?
Re:Wayback Machine director outlines the scale of everyone's... by Dylan16807 on Tuesday October 09, 2018 @08:28PM (#18180966)
Hardware gets cheaper and you can assume a small return on the money, I think. You could also just take it as a guess for 100 years of storage too.