News for hackers. Stuff that matters.
Re:SQLite Is Serverless by tn1 on Sunday January 26, 2020 @12:49PM (#22153449)
The challenge with this is that S3 and friends are object stores, meaning you upload or download the whole file each time. As you can imagine, this will cost you tremendous bandwidth even to insert one row.

Furthermore, it doesn't solve the multiple writers problem, because (afaik) there's no way to lock a file on S3.

Re:SQLite Is Serverless by pm90 on Sunday January 26, 2020 @02:48PM (#22154268)
> As you can imagine, this will cost you tremendous bandwidth even to insert one row.

Is this the cost of network transfers changed by AWS or GCP? What if I'm hosting my app on e.g. EKS or GKE respectively?

Re:SQLite Is Serverless by kondro on Sunday January 26, 2020 @05:14PM (#22155165)
It costs roughly $5/million PUTs and $0.40/million GETs on S3 in addition to the bandwidth and storage you use.

S3 objects are also immutable. Once they’re written, they can’t be updated.

A read-only version of this might be useful, but probably wouldn’t work in-place.

Something that might be of interest is S3 SELECT support that lets you query a single (optionally compressed) CSV, JSON or Parquet file server-side at the same cost of a regular S3 GET.

https://docs.aws.amazon.com/AmazonS3/latest/dev/selecting-co...

And if you really want relational (i.e. JOINs, aggregations and sub-queries) semantics on a bucket full of CSV, JSON, Parquet, ORC or regular-expression-describable files in a cost effective way that has great performance on buckets containing 100’s of TBs of data, definitely look at Athena which is only $5/TB of data scanned during a query.