News for hackers. Stuff that matters.
Re:The Inner Workings of Distributed Databases by moomoo11 on Monday April 17, 2023 @07:33PM (#35608056)
You don’t choose a database to “level up”. It’s a tool.

Use the right tool for the right job.

I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling.

I’ve also migrated from document database to rdbms because the document store didn’t meet our specific requirements.

I wouldn’t just use any random database because I want to appear cool (?) because I know Cassandra or how to use a vector database. That’s not the point.

Re:The Inner Workings of Distributed Databases by tabtab on Monday April 17, 2023 @07:55PM (#35608228)
> I’ve migrated rdbms to wide column databases like Cassandra or dynamo because we had specific requirements that rdbms were not fulfilling.

I'm curious of a common situation. Could RDBMS be improved on that area, or do they inherently lack some necessary property?

I will agree that current RDBMS tend to lack dynamism, and that should be remedied: https://www.reddit.com/r/CRUDology/comments/12ari2l/dynamic_...

Re:The Inner Workings of Distributed Databases by moomoo11 on Monday April 17, 2023 @08:07PM (#35608319)
It’s less to do with rdbms specifics and more to do with things like globally available replication and super low latency.

As for features I mean you could use Postgres and some extensions and you’re good to go. If you want a document db you can just use jsonb type.

You need to know your access patterns and what queries you will be running beforehand when working with something like dynamo. You can also do relational modeling with dynamo following single table design and basically pre join.

Re:The Inner Workings of Distributed Databases by jimbokun on Monday April 17, 2023 @11:29PM (#35609641)
Transactions and joins are the parts that are tricky to implement in a performant way in distributed systems. No SQL systems generally scale well by leaving those features out. Or at least heavily restricting them.
Re:The Inner Workings of Distributed Databases by fzliu on Monday April 17, 2023 @07:59PM (#35608256)
Here's a page on the architecture of a distributed vector database (Milvus) for anybody interested: https://milvus.io/docs/architecture_overview.md
Re:The Inner Workings of Distributed Databases by jimbokun on Monday April 17, 2023 @11:27PM (#35609622)
I want to thank you for the advice to “use the right tool for the job” because it’s certainly not banal, prosaic advice that is invoked in every technology discussion.
Re:The Inner Workings of Distributed Databases by Dylan16807 on Monday April 17, 2023 @11:49PM (#35609760)
> I wouldn’t just use any random database because I want to appear cool

Obviously! "level up" does not imply you used the wrong thing on purpose and you're switching to another solution that's better in every way, as you seem to have read it.

Re:The Inner Workings of Distributed Databases by moomoo11 on Tuesday April 18, 2023 @10:53AM (#35614792)
I’ve met more people who have used the wrong tool for the job than people who are pragmatic and knowledgeable about WHY.
Re:The Inner Workings of Distributed Databases by Dylan16807 on Tuesday April 18, 2023 @02:19PM (#35617762)
Many people use the wrong tool. I don't disagree with that, but also I think you misinterpreted "level up". If they thought the other tool was unqualified better, they would have used it from the start. The reason to ask about the "right time" is because they already know different situations need different tools.