The other option here is using Spanner from Google (https://cloud.google.com/spanner) which now has full PostgreSQL compatibility.

That gets you global scalability with minimal meaningful changes to your application at all. They even made a Rails specific guide for it here

https://cloud.google.com/blog/topics/developers-practitioner...

If you combined that with say Cloud Native Buildpacks (buildpacks.io) and Cloud Run you now have basically an infinitely scalable solution that happens to hook into all of Google’s infrastructure at just the right points that requires almost zero work on your behalf. Plus you get to keep the one command to deploy that a lot of the Rails community is used to from the Heroku days.

No K8s to configure, no servers to maintain, per second billing on your app server, everything autoscales, very little to learn in the way of new technology. You can complicate things at your own pace as far as the rest of “the cloud” goes but there’s no need to do so unless you want to. It’s a pretty sweet deal in terms of cost to benefit ratios.

> full PostgreSQL compatibility

By "full", you mean "some"? From [the documentation] on Spanner's PostgreSQL Interface:

> This means that applications written against the PostgreSQL interface can be readily ported to another PostgreSQL environment, obviously without the non-functional benefits, like scalability and availability, that are unique to Cloud Spanner. However, 100% compatibility with PostgreSQL is not the goal.

> The PostgreSQL interface provides a rich subset of the open-source PostgreSQL SQL dialect, including common query syntax, functions, and operators

It seems like existing simple Postgres applications could switch to Spanner, but it does require running a sidecar if your app wants to continue speaking the Postgres protocol on the wire, instead of integrating a Spanner API client.

When it comes to [datatypes], Spanner's PostgreSQL interface is very limited. According to the docs, only these types are supported:

> bool, bytea (byte array), float8 (64-bit IEEE-754 float), int8 (signed 64-bit int), numeric/decimal, timestampz, text.

Notably missing for [my use-case] are UUID, JSON/JSONB, array types like UUID array, and custom enum types. The indexing support is also quite sparse. Full Postgres compatibility, this is not :(

[the documentation]: https://cloud.google.com/spanner/docs/postgresql-interface

[datatypes]: https://cloud.google.com/spanner/docs/reference/postgresql/d...

[my use-case]: https://www.notion.so

I didn’t know Notion was a Rails product, that’s pretty cool!

Just for disclosure, I used to do a bunch of Rails but I haven’t touched it since the start of covid so it’s applicability to some of what I mentioned was not something I’ve actively built and so I haven’t figured out where all the rough parts are yet because it’s just not a use case of mine these days.

But that second link I mentioned sounds maybe more promising? I initially skimmed it and thought it was there to handle the PostgreSQL to Spanner integration but looking at it a second time it sounds more like native Spanner integration but keeping the otherwise Rails native ActiveRecord experience.

If you’re building against a bunch of PostgreSQL specific functionality then maybe you’re still stuck with the same problem but I don’t think there is a generally speaking more capable publicly available SQL database out there than Spanner than I am aware of is there?

But as a side note if you’re Notion in this situation and not a random small team of a few people I don’t imagine you would have much of a problem reaching out to the GCP team to explain your current set up / requirements, they would be more than happy to give you a custom roadmap to migrate to Spanner, they have entire teams dedicated to exactly those kinds of projects.

Notion isn't a Rails product - but we do use Postgres as our primary datastore.

The difference being if you want multi-region availability with Spanner, be ready to pay minimum $2k per month per region your data is in ($3/hr * 24 hours * 31 days)

I think there is probably an argument to be made that if that’s a scary and unreasonable sounding number for your use case then it’s not the right tool for the job and maybe you might want to consider PostgreSQL knowing that you can migrate later if needed or if you don’t have a lot of many to many data relations then even a document database might get you that kind of scalability benefits but at a fraction of the cost.

Yes, lets throw data integrity out the window and make schemas changes painfully tedious and hard.

Does DynamoDB give transactions, rollbacks, joins?

Yes, you can apply a set of updates atomically so they all fail or none, with conditions. No save points or nested transactions, though. And the transactions need to be grouped into a single request.

Joins, no. Most of the NoSQL systems distribute data by partitioning them on the primary key, and give you scalability by not doing any work across partitions. Which means that traditional joins won’t be supported. You can either join the application, or maintain a secondary schema that has your joined data organised in the access pattern you plan to use.

Google DynamoDB Alex DeBrie or Rick Houlihan for a ton of tutorials on how to restructure data for partitioned/distributed NoSQL systems.

and max 100 keys in a transaction or whatever the limit is