Re:S3 is files, but not a filesystem
by wenc on Sunday March 10, 2024 @01:35PM
(#39660908)
S3 is really high latency though. I store parquet files on S3 and querying them through DuckDB is much slower than file system because random access patterns. I can see S3 being decent if it’s bulk access but definitely not for random access.
This is why there’s a new S3 Express offering that is low latency (but costs more).
This is why there’s a new S3 Express offering that is low latency (but costs more).