News for hackers. Stuff that matters.
Re:S3 Express Is All You Need by throwitaway222 on Tuesday November 28, 2023 @03:28PM (#38450822)
I don't understand why EFS never gets major shout outs - it's way better than S3: systems can mount it as a drive, shared across systems, already has had super low latency... Not sure what s3 express is really useful for if EFS already exists.
Re:S3 Express Is All You Need by candiddevmike on Tuesday November 28, 2023 @03:30PM (#38450844)
EFS is really expensive and has terrible latency with small files in my experience
Re:S3 Express Is All You Need by dekhn on Tuesday November 28, 2023 @04:58PM (#38451919)
When you set up EFS did you maximize the IO settings?

Before doing that it was unacceptably slow. After doing that it was unacceptably expensive.

Re:S3 Express Is All You Need by huntaub on Tuesday November 28, 2023 @03:40PM (#38450984)
Note that EFS One Zone is priced the same as S3 Express One Zone with similar latency. One isn't better or worse than the other, it only depends on what kind of access your application needs.
Re:S3 Express Is All You Need by brazzledazzle on Tuesday November 28, 2023 @03:33PM (#38450878)
Yeah the main reason is that it's incredibly expensive. You can improve performance by allocating ahead of time but NFS has never been at its best when working with a bunch of tiny files.
Re:S3 Express Is All You Need by richieartoul on Tuesday November 28, 2023 @03:35PM (#38450913)
Do you have any more details you can share about the performance of EFS? I've never met anyone who has actually used it in anger.
Re:S3 Express Is All You Need by gchamonlive on Tuesday November 28, 2023 @03:42PM (#38451002)
Throughput scales with the amount of data in it, it is in the docs. So depending on the application, even if latency is better, the speeds are atrocious at lower volumes of persisted data.
Re:S3 Express Is All You Need by saddlerustle on Tuesday November 28, 2023 @04:52PM (#38451813)
That’s not true anymore with EFS Elastic Throughput
Re:S3 Express Is All You Need by a2tech on Tuesday November 28, 2023 @05:23PM (#38452244)
Yes, I built a moderately large system on it that used lots of small shared files. The performance was fairly terrible. There's weird little niggles with it--we had random slowdowns, throughput issues, and things just didn't work quite right.

It was an ok solution for what we were doing, but several times I came really close to just dumping it and standing up an NFS server using EBS volumes.

I also used it a couple of times to store webroots and that was a complete disaster with systems that had lots of small files (Drupal I'm looking at you).

Re:S3 Express Is All You Need by sparrc on Tuesday November 28, 2023 @04:05PM (#38451301)
In my experience the biggest drawback with EFS is startup time for systems that mount it in.

For example a container or EC2 instance might only need a tiny bit of your storage and with s3 can just download what it needs when it needs it.

As opposed to EFS where the container or instance needs to load in the entire datastore on startup which can add minutes to startup time if the EFS drive is large.

Re:S3 Express Is All You Need by dpedu on Tuesday November 28, 2023 @04:13PM (#38451396)
My understanding is that EFS is exposed as an NFS share. I haven't used it personally, but NFS mounting is generally fast, nearly instant. What does "load in the entire datastore" mean?
Re:S3 Express Is All You Need by ericpauley on Tuesday November 28, 2023 @04:39PM (#38451676)
EFS mounting is definitely nearly instant. I use it constantly.
Re:S3 Express Is All You Need by dekhn on Tuesday November 28, 2023 @07:36PM (#38453807)
Many servers start up, load a ton of data from storage into RAM, and then happily serve that data for a long time. The latency of the server when starting up before it can service its first request is entirely based on the throughput of the data load.

Often these servers will load 128+GB of data into RAM (crazy, huh?) and even if you have 1GB/sec it's still two minutes for the server to start up.

Re:S3 Express Is All You Need by dpedu on Wednesday November 29, 2023 @12:57PM (#38462702)
1GB/s is the same speed from EFS or S3.
Re:S3 Express Is All You Need by dekhn on Wednesday November 29, 2023 @01:22PM (#38463048)
I didn't say otherwise.
Re:S3 Express Is All You Need by yeeeloit on Tuesday November 28, 2023 @03:53PM (#38451153)
I wonder if Mountpoint for S3 along with this new Express option makes it a direct competitor to EFS for some use cases.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountp...

Re:S3 Express Is All You Need by tneely on Tuesday November 28, 2023 @04:31PM (#38451591)
I'm quite curious about this too - both from a cost and performance perspective. If S3 Express is close enough to EFS on these metrics, then I'd say it wins out due to the sheer ubiquity and portability of S3 these days.
Re:S3 Express Is All You Need by halz on Wednesday November 29, 2023 @04:31AM (#38457331)
Looks like support for S3 Express was merged in with version 1.30 just a few hours ago https://github.com/awslabs/mountpoint-s3/pull/642
Re:S3 Express Is All You Need by toomuchtodo on Tuesday November 28, 2023 @03:44PM (#38451048)
EFS exists if you don't care much about spend and performance while having to forklift a POSIX compliant use case into AWS for persistent data.
Re:S3 Express Is All You Need by a2tech on Tuesday November 28, 2023 @05:27PM (#38452295)
Thats basically how we were using it. It could have been worse.
Re:S3 Express Is All You Need by notatoad on Tuesday November 28, 2023 @06:36PM (#38453199)
i'm sure there's some cases where mounting storage as a disk is desirable, but from my perspective "systems can mount it as a drive" is a negative, not a positive.

treating storage as an application-controlled thing that doesn't need systems management is a good thing. i want "put this file in this spot" logic in my application code, not "put this file in this spot on the filesystem, and hope that location is backed by the correct storage layer"