News for hackers. Stuff that matters.
Re:Internet Archive Infrastructure by janjim on Tuesday March 02, 2021 @04:33AM (#26313489)
Curious about the cost. Does that already include manpower and various acquisition cost of constructing their internal network (hardware, fiber link between site)?

I guess the biggest downside is the speed of scaling that they can do. As it is limited by how fast they can purchase and install new storage device. But with the use case of Internet Archive, that shouldn't matter much.

Re:Internet Archive Infrastructure by Nextgrid on Tuesday March 02, 2021 @05:03AM (#26313656)
I've seen this argument a lot but I'm not sure how well it holds. The price for performance ratio on cloud providers is so poor that you can overprovision in advance (to mitigate the extra delay involved in adding extra hardware) and still come out ahead.

Also, bare-metal doesn't necessarily mean owning the hardware. You can rent it too. There are providers that provide bare-metal in one-click and sometimes available within minutes.

Re:Internet Archive Infrastructure by Sebb767 on Tuesday March 02, 2021 @09:10AM (#26315463)
> The price for performance ratio on cloud providers is so poor that you can overprovision in advance (to mitigate the extra delay involved in adding extra hardware) and still come out ahead.

It really depends on what scale you're talking about. When you're a startup and suddenly land on the front page of HN, you might need 100x or 1000x your current capacity - in which case AWS will be useful to no end.

If, on the other hand, you're an established name with quite a bit of traffic already and the maximum uptick you will reasonably experience is 2x-3x, the argument holds far less water.

Re:Internet Archive Infrastructure by ddorian43 on Tuesday March 02, 2021 @10:05AM (#26316022)
Feel free to point to these cases where people scaled 1000x when they hit the HN frontpage. Especially their database.
Re:Internet Archive Infrastructure by ddorian43 on Tuesday March 02, 2021 @07:30AM (#26314598)
He said his storage pricing is 2-5x cheaper than google archive line. That is $1.2/3x= $0.4/TB/month. Compare that against $20/month S3. He has 50x less cost. He can afford to overprovision.
Re:Internet Archive Infrastructure by bnewbold on Tuesday March 02, 2021 @02:53PM (#26319715)
If you are interested in cost modeling for long-term digital preservation, check out this blog series: https://blog.dshr.org/2019/02/economic-models-of-long-term-s...