NASA to launch 247 petabytes of data into AWS, but forgot about egress costs

Re:NASA to launch 247 petabytes of data into AWS, but forgot... by Spooky23 on Thursday March 19, 2020 @01:00PM (#22629362)

Sounds like there is a bigger story there and it's probably a managed SAN.

I've operated pretty significant government shared infrastructures like this in the past... we were offering fast, flash-cached disk in 2010 for about $5,000/TB. $10k/TB is not unreasonable for highly available Tier-1 storage for something like SAP, especially in that era where you couldn't use all flash in most case.

Today, cost structures can be very different. You can land high-iop storage for a fraction of the cost without the overhead of a big SAN. If you need capacity focused storage, that is also much cheaper.

An agency like NASA gets hosed on services, and cloud is no different. AWS is probably a net savings for operational workloads whose characteristics are known. Backup is a no-brainer. But for a high-volume, operationally highly variable thing like a public archive of data, AWS a square peg in a round hole because of the metered access.

[ Parent | on HN ]

Re:NASA to launch 247 petabytes of data into AWS, but forgot... by topkai22 on Thursday March 19, 2020 @11:17PM (#22634358)

I’m sure that $10k/terabyte quote was complete overkill for what we needed- but that’s what the stove piped storage org was offering, and it killed the project we were working on.

[ Parent | on HN ]

Re:NASA to launch 247 petabytes of data into AWS, but forgot... by 2J0 on Friday March 20, 2020 @12:38PM (#22638955)

I hope you can correct my numbers but I am pretty sure this is within the same decimal order of magnitude :

If 1-2TB drives were handily $1k in 2010 (2005 $1K hot you 128GB 15KRPM)

and your array set is at least R10,

already raw storage is approaching half of ten thousand dollars.

And this ignores controllers, cabling and chassis.

And this is before we look at our storage software licenses.

Is backup, point in time SLA, replication and availability in this budget?

[ Parent | on HN ]

Re:NASA to launch 247 petabytes of data into AWS, but forgot... by topkai22 on Monday March 23, 2020 @12:34AM (#22661297)

I wasn't really sure what they pitched us technically, but your pitch sounds reasonable. It was also complete overkill- we were hosting read only static images (map tiles). Azure and AWS were less than $300/TB/Year at the time, and their triple replication was more than what we needed availability wise.

[ Parent | on HN ]

Re:NASA to launch 247 petabytes of data into AWS, but forgot... by solarkraft on Friday March 20, 2020 @07:40AM (#22636739)

Maybe I'm missing vital context info here: Why didn't you go with an alternative?

[ Parent | on HN ]

Re:NASA to launch 247 petabytes of data into AWS, but forgot... by topkai22 on Monday March 23, 2020 @12:22AM (#22661249)

Because the storage group refused to sign off on a cheaper solution with lower specs (I don't know why) and acquisitions in the government is a mess so going outside would have tied up one of our primary constraints (the tech lead) more than it was worth.

The overall system ended up with worse capabilities than it should have had, but it did ship.

[ Parent | on HN ]