News for hackers. Stuff that matters.
Re:S3 is files, but not a filesystem by CobrastanJorji on Sunday March 10, 2024 @03:39AM (#39657417)
They are necessary because as soon as someone decides that S3 is a filesystem, they will look at the other cloud "filesystems," notice that S3 is cheaper than most of them, and then for some reason they will decide to run giant Hadoop fs stuff on it or mount a relational database on it or all other manner of stupidity. I guarantee you S3's customer-facing engineers are fielding multiple calls per week from customers who are angry that S3 isn't as fast as some real filesystem solution that the customer migrated from because S3 was cheaper.

When people decide that X is a filesystem, they try to use it like it's a local, POSIX filesystem, and that's terrible because it won't be immediately obvious why it's a stupid plan.

Re:S3 is files, but not a filesystem by albert_e on Sunday March 10, 2024 @04:41AM (#39657646)
If a customer makes an IT decision as big as running Hadoop or RDBMS with S3 as storage ... but does not consult at least a Associate level AWS Certified architect (who are doke a dozen) for at least one day worth of advice which is probably a couple of hundred dollars at most ...

Can we really blame AWS?

I am sure none of official AWS documentations or examples show such an architecture.

----

Amazon EMR can run Hadoop and use Amazon S3 as storage via EMR FS.

"S3 mountpoints" are a feature specifically for workloads that need to see S3 as a file system.

For block storage workloads there is EBS and EFS and FSx that AWS heavily advertises.

Re:S3 is files, but not a filesystem by albert_e on Sunday March 10, 2024 @10:06AM (#39659185)
*dime a dozen

(Apologies for typos. The "noprocrast" setting sometimes locks us out of HN right after submitting a comment. And it is now too late, not editable)