Is this the cost of network transfers changed by AWS or GCP? What if I'm hosting my app on e.g. EKS or GKE respectively?
S3 objects are also immutable. Once they’re written, they can’t be updated.
A read-only version of this might be useful, but probably wouldn’t work in-place.
Something that might be of interest is S3 SELECT support that lets you query a single (optionally compressed) CSV, JSON or Parquet file server-side at the same cost of a regular S3 GET.
https://docs.aws.amazon.com/AmazonS3/latest/dev/selecting-co...
And if you really want relational (i.e. JOINs, aggregations and sub-queries) semantics on a bucket full of CSV, JSON, Parquet, ORC or regular-expression-describable files in a cost effective way that has great performance on buckets containing 100’s of TBs of data, definitely look at Athena which is only $5/TB of data scanned during a query.
Furthermore, it doesn't solve the multiple writers problem, because (afaik) there's no way to lock a file on S3.