News for hackers. Stuff that matters.
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by olivermuty on Monday May 20, 2024 @03:06AM (#40413051)
Tape robot time is NOT measured in minutes but hours. Also you wouldn't pull individual files like this out of tape.

Tape is for long term sunk storage, not cold infrequent access like a youtube video.

I know aws glacier has an "expedited retrieval time" of 1-5 minutes, but that is not how typical tape setups work. Frankly I would be very interested in what actually hides behind that product.

Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by joking on Monday May 20, 2024 @03:12AM (#40413077)
You can use big disk, but not be able to access all the files on the disk with the same frequency, so you have 20% of the disk dedicated to hot storage, and 80% of it to cold storage. Cold storage access is queued , so the 1-5 minutes can come from there.
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by olivermuty on Monday May 20, 2024 @03:16AM (#40413104)
Yeah I looked closer and I think they are basically just packaging up some kind of offering on top of spinning rust for the two first glacier tiers.
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by swiftcoder on Monday May 20, 2024 @10:06AM (#40415607)
Glacier is not actually tape, the fancy tape-robot videos nonwithstanding. Most of it is just regular old S3 running on outdated storage hardware
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by remram on Monday May 20, 2024 @06:54PM (#40421510)
Interesting. Why does it take hours to retrieve then? Network/disk bandwidth really is that bad? Also do you have a source for this?
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by magarnicle on Monday May 20, 2024 @08:04PM (#40422266)
To discourage access, maybe? That pushes you to use more expensive options.
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by remram on Tuesday May 21, 2024 @01:05PM (#40431040)
This doesn't hold, not having the cheap option encourages you to use the expensive option even better...
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by swiftcoder on Tuesday May 21, 2024 @04:29AM (#40425571)
It's mostly just bandwidth prioritization, slotting large transfers in when there is excess bandwidth.

You can tell this is the case for at least the flexible retrieval tier, because small objects can be returned in a few minutes, whereas larger requests take hours - if the files were actually on a tape drive somewhere, small requests couldn't be fulfilled dramatically faster than large ones, given that tape has shitty random-access performance.

(I used to work on a downstream team at AWS)

Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by remram on Tuesday May 21, 2024 @09:39AM (#40428369)
I believe you, I can see how it would make sense that AWS would create a tier to exploit the spare capacity in S3 disk bandwidth, just like they did for EC2 spare VM capacity with spot instances. Still it doesn't make intuitive sense to me how the performance AND the price can be so far. That's why I'd love a longer write-up if you know of any.

It's also weird that the retrieval gives you a regular fast S3 object you can then access. Given that it's already on that hardware, is a copy even happening?

Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by surajrmal on Monday May 20, 2024 @11:53AM (#40416844)
Google gave up on tape a while back. Latest Google search indicates it is only used for air gapped backups. I don't OP was suggesting using tape though, especially with technologies such as hybrid SMR.
Re:Ask HN: Video streaming is expensive yet YouTube "seems" ... by PhasmaFelis on Monday May 20, 2024 @12:21PM (#40417116)
That's all true, but I don't think anyone mentioned tape storage.