News for hackers. Stuff that matters.
Re:AWS cancels serverless Postgres service that scales to zero by jvanderbot on Wednesday January 10, 2024 @09:25AM (#38939303)
Honest question, why not just use dynamodb? If I'm already mostly server less and AWS, dynamodb seems like a natural choice for a persistent store in a world where cost and zero-scaling is important.
Re:AWS cancels serverless Postgres service that scales to zero by hibikir on Wednesday January 10, 2024 @10:13AM (#38939753)
Dynamo is NoSQL, with all the architectural tradeoff that brings: Say goodbye to joins! So depending on how your data looks like, dynamo can an pretty bad fit. Do not run your reporting engine in Dynamo.

It gets worse than that though, as Dynamo's usage on large tables depends on its hottest shard, and other than setting primary keys, you have no control over how it shards. The story of people moving to dynamo and touting its advantages, just to move out a year later because the shape of their data makes dynamo prohibitively expensive as the data grows is pretty common. I once worked at a place where they stored historical data in dynamo, leading to a substandard hash key. Most of the time the db was idle, but when it wasn't, the sharding scheme made them pay a good 500x capacity than it was actually in use, because while a shard was red hot, others were completely idle. The monthly price for this relatively unimportant feature ended up being higher than the office's rent, in San Francisco.

If your table's key is UUIDs, and the chances of querying one record or another is almost perfectly flat, then sure, dynamo away! But if you walk away from it very best use case, and you suddenly start having any amount of data... dynamo can become really, really expensive. I'd not say you should never use it, but it's a really scary first place to go, precisely if you care about costs.

Re:AWS cancels serverless Postgres service that scales to zero by wmfiv on Wednesday January 10, 2024 @09:43AM (#38939445)
Yeah agreed. It's about tradeoffs. If you want scale to zero with low latency DynamoDB already exists and provides that. The tradeoff is you give up all the nice features of a RDMS.
Re:AWS cancels serverless Postgres service that scales to zero by danielbarla on Wednesday January 10, 2024 @09:59AM (#38939584)
DynamoDB is _fantastic_ for certain scenarios. But, you also need to know your access patterns upfront and more or less end up hardcoding your indexes into the "table design". It's not a 1:1 replacement for a typical relational DB's use cases.
Re:AWS cancels serverless Postgres service that scales to zero by qvrjuec on Wednesday January 10, 2024 @09:28AM (#38939318)
Because you need SQL. Dynamo can be quite powerful for querying if you use all of its features but it won't be as flexible as just using SQL
Re:AWS cancels serverless Postgres service that scales to zero by belter on Wednesday January 10, 2024 @10:07AM (#38939669)
Almost any SQL workload you can move to DynamoDB
Re:AWS cancels serverless Postgres service that scales to zero by dragonwriter on Wednesday January 10, 2024 @11:46AM (#38940521)
DynamoDB lacks consistency (in the ACID sense, it has been it in the CAP sense which is more related to Atomicity in ACID), so you can move an SQL workload to DynamoDB by moving the consistency parts out of the DB into the code of any application that hits the DB.
Re:AWS cancels serverless Postgres service that scales to zero by belter on Wednesday January 10, 2024 @01:03PM (#38941290)
You hardly get the ACID on a relational database :-)

"When is "ACID" ACID? Rarely." - http://www.bailis.org/blog/when-is-acid-acid-rarely/

Re:AWS cancels serverless Postgres service that scales to zero by dragonwriter on Wednesday January 10, 2024 @01:28PM (#38941580)
That's an article about isolation, not consistency; and even there, both the RDBMS’s relevant to thus discussion (those on Aurora Serverless that it is suggested in this subthread be replaced with DynamoDB) supported serializable isolation even in their current versions at the time of that article a decade ago.

So your post is both irrelevant as a response to mine about consistency and more generally irrelevant to the entire discussion here.

Re:AWS cancels serverless Postgres service that scales to zero by belter on Wednesday January 10, 2024 @01:43PM (#38941724)
The article is about ACID, both consistency and isolation not just about consistency and you can't isolate them (pun intended). The underlying of the discussion was not on consistency, instead on the possibility of using DynamoDB where you would use a typical relational engine.

From the article: "The textbook definition of ACID Isolation is serializability (e.g., Architecture of a Database System, Section 6.2), which states that the outcome of executing a set of transactions should be equivalent to some serial execution of those transactions. This means that each transaction gets to operate on the database as if it were running by itself, which ensures database correctness, or consistency."

" This means that each transaction gets to operate on the database as if it were running by itself, which ensures database correctness, or consistency. A database with serializability (“I” in ACID), provides arbitrary read/write transactions and guarantees consistency (“C” in ACID), or correctness, of the database. Without serializability, ACID, particularly consistency, is generally not guaranteed"

I am of course ignoring the Consistency you are certainly aware that exist in DynamoDB with Strong Consistency and DynamoDB Transactions.

Moving consistency management outside of DynamoDB as you propose doesn't circumvent the CAP theorem's limitations.

Re:AWS cancels serverless Postgres service that scales to zero by e12e on Wednesday January 10, 2024 @10:25AM (#38939835)
https://aws.amazon.com/blogs/database/how-to-determine-if-am...

> Some unsuitable workloads for DynamoDB include:

> Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome.

> Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally require distribution and the joining of fact and dimension tables that inherently provide a normalized (relational) view of your data.

> Binary large object (BLOB) storage. DynamoDB can store binary items up to 400 KB, but DynamoDB is not generally suited to storing documents or images. A better architectural pattern for this implementation is to store pointers to Amazon S3 objects in a DynamoDB table.

Re:AWS cancels serverless Postgres service that scales to zero by belter on Wednesday January 10, 2024 @01:20PM (#38941472)
I said moving the workload to DynamoDB, not doing the same query pattern with DynamoDB ... :-)

For each of these workloads you switch the query pattern to accommodate for DynamoDB and provide a solution for your workload and/or app. That is actually the secret of NoSQL. You do the work upfront.

So your reading of the recommendations is correct on the surface but incorrect on the fundamental usage aspect of it. Lets have a look at each:

> Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome.

A: Dont do this. Don't do ad hoc query access. Define a series of access patterns and lay out your data to support them. You dont get ad hoc query access for Netflix, Amazon or your Airline Travel website...

> Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally require distribution and the joining of fact and dimension tables that inherently provide a normalized (relational) view of your data.

These are not most SQL Patterns. This is OLAP and was always or should be done with MPP systems, not with your relational database like your SQLServer or your Oracle. So this is going on an edge...

> Binary large object (BLOB) storage. DynamoDB can store binary items up to 400 KB, but DynamoDB is not generally suited to storing documents or images. A better architectural pattern for this implementation is to store pointers to Amazon S3 objects in a DynamoDB table.

Your Relational Database will not store BLOBs more efficiently than S3 anyway....

Re:AWS cancels serverless Postgres service that scales to zero by Ao7bei3s on Wednesday January 10, 2024 @03:29PM (#38942660)
DynamoDB can't even represent SELECT * FROM items LIMIT 25 OFFSET 100. It's just not designed for that. It's not meant to be a relational DB replacement.

How would you do it? Assume we want proper pagination, and not rewrite the app for cursor based "Load more" style pagination. Why? Because the React Admin provider API insists. https://github.com/marmelab/react-admin/issues/1510

Re:AWS cancels serverless Postgres service that scales to zero by belter on Wednesday January 10, 2024 @04:59PM (#38943887)
Your example itself shows why you would not want to do this. The database still has to read through all the rows up to the OFFSET point...

You are not supposed to do the same query patterns. My argument is that you can substitute your relational database, by changing the app and the layout of the data to match the proper patterns for DynamoDB.

"Migrating to DynamoDB from a relational database" - https://docs.aws.amazon.com/amazondynamodb/latest/developerg...

""How to model one-to-many relationships in DynamoDB" - https://www.alexdebrie.com/posts/dynamodb-one-to-many/

Re:AWS cancels serverless Postgres service that scales to zero by earthling8118 on Thursday January 11, 2024 @11:19AM (#38954284)
If you place arbitrary restrictions upon it then of course it won't fit your model. Dynamo can handle pagination easily by passing in the last seen ID in the request. We used IDs that were lexigraphically sortable by time which has some nice properties such as never losing your position due to extra items being inserted in the middle of your query. Offset/limit aren't fun to work with.

You can go further by using the streams feature to dump your data into an analytical database for your querying needs.

Re:AWS cancels serverless Postgres service that scales to zero by pid-1 on Wednesday January 10, 2024 @10:00AM (#38939592)
That's like telling people to use S3, those are completely different storage systems.