News for hackers. Stuff that matters.
Re:How and why the Relational Model works for databases by oblio on Monday January 17, 2022 @05:20AM (#29965114)
You've just torn apart most NoSQL databases in a very short comment :-))
Re:How and why the Relational Model works for databases by neuronic on Monday January 17, 2022 @05:34AM (#29965200)
So this is just an observation and not a judgment.

With only 4 years of professional experience I have never worked on a MongoDB project in which MongoDB somehow wasn't an issue. The proposed solutions merely being "improve the indices" or "scale the cluster", often without clearly defining what's going on.

So granted I have a very limited experience, it's baffling to me mainly in comparison to the PostgreSQL projects I worked on. They definitely also had problems but these problems were clearly defined even if resolution wasn't quick or easy. It was usually an out-of-date or generally messy schema that was causing issues and folks usually were able to clearly define the schema problems.

Re:How and why the Relational Model works for databases by emptysongglass on Monday January 17, 2022 @05:56AM (#29965330)
I hear this about MongoDB (that it's a plague and you should stay far away) all the time but DynamoDB, which is also NoSQL? One of AWS' finest products; just launching an app on AWS? Use DynamoDB! So on and so forth ad nauseum.

Why is DynamoDB the bee's knees but MongoDB is a thing to be despised?

Re:How and why the Relational Model works for databases by oblio on Monday January 17, 2022 @06:03AM (#29965363)
1. One has a better marketing machine than the other?

2. DynamoDB is pushed in slightly more balanced ways than Mongo was, at the start Mongo was supposed to be the second coming of the (database) Messiah.

3. I don't believe DynamoDB has defaults that lose your data.

Re:How and why the Relational Model works for databases by ttfkam on Monday January 17, 2022 @11:59AM (#29968604)
Mongo works really well for backends to test your UI against in coding bootcamps. Dead simple CRUD storage and access model. As a mock React data store, it'd be hard to find simpler.

DynamoDB is a managed service that scales far beyond Mongo and truthfully far beyond what any of us here would be seriously discussing.

With regard to complex schemas and related data, both DynamoDB and Mongo are harder to deal with once you've past a trivial size. If all you need is basic CRUD with limited or no joins, Mongo could suffice to a large deployment, but if you're mostly accessing by primary key, DynamoDB will soundly eat its lunch performance-wise and for cheaper.

If you need to join related data (or just find them convenient), relational models work famously well, and usually up the point where you have 10 million simultaneous users.

Mongo more and more is being relegated to niches where the problem just happens to exactly fit Mongo's feature list, which unfortunately for Mongo encompasses ever-narrowing gaps in the other options' offerings.

Mongo can't match the scale or economy of DynamoDB and can't match the flexibility of relational. That's why fewer and fewer treat it as their go-to database nowadays.

Re:How and why the Relational Model works for databases by jd_mongodb on Tuesday January 18, 2022 @05:36AM (#29977306)
Your comment that "fewer and fewer treat it as their go-to database nowadays" seems odd given the continued growth of MongoDB.

https://db-engines.com/en/ranking_trend/system/MongoDB

https://www.macrotrends.net/stocks/charts/MDB/mongodb/revenu...

https://seekingalpha.com/article/4473768-mongodb-mdb-stock-s...

Re:How and why the Relational Model works for databases by to11mtm on Monday January 17, 2022 @09:26AM (#29966814)
Dynamo seems nice for some very specific use cases, but there are others where it falls short.

There's limits on how big your data can be, which is a little annoying if you want to use it to store a couple larger things alongside all your small data.

The C# API is absolutely painful IMO, and makes it easy for new developers to use it in a way that you would have been better off grabbing a different technology.

I suppose one could argue S3 would be the better choice for large payloads, OTOH I'm a fan of minimizing potential points of failure so if we already are using DynamoDB I'd rather not toss in an additional S3 integration that could break or need maintenance later.

Re:How and why the Relational Model works for databases by ttfkam on Monday January 17, 2022 @11:44AM (#29968432)
1. Below 25GB and the read/write threshold of most apps, DynamoDB is free.

2. DynamoDB is a 100% managed service. No instances to wrangle. No manual partitioning. Just define your table's name, the table's partition key, and an optional sort key, and you're off to the races! It just works.

If you have a serverless environment like lambdas with API Gateway or AppSync, it can scale pretty much to the extent of your business model rather than some fixed limit.

BUT for data schemas beyond the most trivial, it can easily be more complex to deal with than a relational database. Whereas a relational database usually aims for normalization where no data is duplicated and foreign keys keep things straight, DynamoDB works best with a denormalized data set. No joins. Ever. Schema integrity is your problem, not the database's. In the deal though, you get a database engine that can scale effectively infinitely.

In other words, storage is cheap, but access is expensive, so data duplication is pretty much encouraged in DynamoDB for the sake of speed. You aim for getting everything you need in a single entry or sequential row iteration.

When we use a relational database and run into problems, we run EXPLAIN and EXPLAIN ANALYZE to figure out the query plan, so we can optimize. SQL is a 4th generation, declarative language that describes WHAT data you want, not HOW you get it.

DynamoDB in the larger sense is at the level of EXPLAIN output. It is 100% HOW to get data. The WHAT is at application level and implemented by you in code.

If EXPLAIN output makes no sense to you, then DynamoDB probably isn't for you either unless it's a trivial app/data set.

But then again, if it's a trivial app/data set, literally anything can work. An O(n!) algorithm is perfectly reasonable given a small/simple enough data corpus and large enough computing resources. It's when the data set gets slightly larger that decisions become important.

But when things are very small/simple, it's often hard to argue with fast+free. Those are the sweet spots for DynamoDB: very small/simple and the mind-bogglingly humongous. For everything in the middle, relational databases work wonderfully and are much easier to work with, especially for non-trivial data sets.

Re:How and why the Relational Model works for databases by buzzwordninja on Monday January 17, 2022 @10:56AM (#29967835)
For one thing DynamoDB is Amazon scale whereas Mongo is merely web-scale.
Re:How and why the Relational Model works for databases by skeeter2020 on Monday January 17, 2022 @11:29AM (#29968242)
But is it shardable? ;)
Re:How and why the Relational Model works for databases by redwood on Monday January 17, 2022 @02:44PM (#29970880)
DynamoDB is a key value store so forces you into an up-front waterfall design model where you can't easily evolve your schemas. This has massively weakened the scenarios where it can be used but there's a survivor bias where if that's all you need you're really dealing with a simple use case that's easy to encapsulate and reason about
Re:How and why the Relational Model works for databases by uvdn7 on Monday January 17, 2022 @09:53AM (#29967087)
DynamoDB is evolved from Dynamo, described in details in their SIGOPS’07 paper. Amazon uses Dynamo internally for e.g. shopping charts. There’s nothing wrong with a distributed key/value store. It’s just that I think Dynamo started off being super reliable, and not overpromising …
Re:How and why the Relational Model works for databases by skeeter2020 on Monday January 17, 2022 @11:28AM (#29968234)
Don't worry; for every misuse of a NoSQL database when they should have used a relational DB there's a solution misusing a relational DB for a queue, event bus, etc. Nature likes a balance!
Re:How and why the Relational Model works for databases by kreeben on Monday January 17, 2022 @12:01PM (#29968631)
Would you remind a dumb-wit like myself why that's a problem, when the API of a relational db is virtually perfect for a queue or event bus? Push, pop, look-but-don't-touch, data available through the most scrutinized and battle-tested query language ever, what else do you need?
Re:How and why the Relational Model works for databases by bshipp on Monday January 17, 2022 @12:20PM (#29968896)
until Postgres added JSON fields I would have retorted that nosql is really handy for scraping purposes, when you just want to grab and store the collected data, which you then parse with another specific program into a relational database later.

I've learned this by doing it the opposite way for years. however, website schemas can frequently change, which ruins the database schema if you're immediately parsing into a normalized structure. In other words, it's brittle.

noSQL (or JSON fields) allow one to store unstructured data which is more forgiving when a schema changes (I.e. change the spelling of a dictionary key, etc.).

Re:How and why the Relational Model works for databases by vaughan on Monday January 17, 2022 @07:26PM (#29973337)
What you need is a versioned mapping layer from the scraped data into your own schema.

The problem is when you need to update your schema to support data/relations not available in your own schema.

The problem with SQL is that schemas are so sticky and hard to change. People are always hesitant to change their schemas because schema changes themselves are difficult and then you have to update a bunch of backend code, and then your frontend code maybe does strange denormalized things, and then everything breaks.

I've been thinking about this for a while, and I think what is needed is a visual tool that can show all dependencies of a data schema element (backend, frontend), so that schema changes are easier to make.

All the layers (db access, api, cache, frontend data stores/caches/frameworks) in modern architectures make it virtually impossible to modify a schema without causing chaos. The solution is keeping the data model and query interface as close as possible throughout the entire stack. For example you should never write any manual data manipulation code (e.e. `people.map(p => p.full_name = p.firstName + p.lastName)` unless this is able to be traced through the entire system. A monorepo, typed ORM, refactoring tooling (e.g. IDE) can help, but its usually never setup well enough or integrates close enough with the db.

Re:How and why the Relational Model works for databases by oblio on Monday January 17, 2022 @01:28PM (#29969891)
Well, technically you could have done this since forever in an RDBMS. Key value table, have the table be a blob (or just varchar if a string).

Then yes, you'd have to parse it yourself, but you'd be doing that anyway with JSON fields, more or less.

Re:How and why the Relational Model works for databases by bshipp on Monday January 17, 2022 @02:33PM (#29970760)
Indeed, you are correct. A lot of these lessons came hard-earned from doing it the wrong way first.

I thought I was being such a good little programmer by normalizing my data as soon as possible. All it did was provide me job security (at the cost of headaches) every time the schema changed.

Re:How and why the Relational Model works for databases by timando on Tuesday January 18, 2022 @06:10PM (#29987479)
Why does PostgreSQL include LISTEN/NOTIFY if I'm not supposed to use it for events?
Re:How and why the Relational Model works for databases by cle on Monday January 17, 2022 @09:45AM (#29966986)
NoSQL databases are mostly useful for performance reasons. You can have the most beautifully modeled relational database schema but if it performs/scales outside your tolerance bounds, then it's useless.

IMO, this is the main reason we have so many database technologies: they make important performance tradeoffs.

Re:How and why the Relational Model works for databases by EMM_386 on Monday January 17, 2022 @11:54AM (#29968536)
It isn't that difficult to optimize relational SQL databases. I've dealt with databases with billions of rows in hundreds of tables and the company's initial thought was to flatten the schema due to perf. complaints.

We got queries down from 30+ seconds to 5 ms simply by properly indexing, defragmentation, analyzing query plans, SQL Stored Procedures, etc.

I see a lot of complaints from developers claiming they have to join a "million row table to million row table" and reports are slow, and this gets blamed on the DB. These should not be slow outside of how much bandwidth is being pushed over the wire, which is often exactly what the problem is. They just didn't see it.

Re:How and why the Relational Model works for databases by bshipp on Monday January 17, 2022 @12:22PM (#29968930)
I'm a huge supporter of PGhero (https://github.com/ankane/pghero) for this reason. Sometimes a single index is all that's required, but if DB design isn't your forte it's hard to know where to put it.
Re:How and why the Relational Model works for databases by cle on Monday January 17, 2022 @03:03PM (#29971106)
When you start seeing devs make the same mistakes over and over again with relational databases in certain contexts, that increase the costs of operational maintainance and scaling (e.g. by requiring them to sit around analyzing query plans, lock contention, sharding, etc.), then it makes sense to consider building a DB with constraints that prevents those mistakes and ensures that every DB instance can meet those requirements with a lower overall cost. That is how I view NoSQL, graph databases, etc.

It is similar IMO to the diversity in programming languages. They are all Turing complete, at the end of the day. The difference is in the patterns they encourage and the constraints they impose.

Re:How and why the Relational Model works for databases by vaughan on Monday January 17, 2022 @07:38PM (#29973440)
I've always felt a more modular SQL database system is needed. SQL dbs are too black box. You can't interact with the query planner. NoSQL essentially is doing just this. SQL dbs have great underlying layers such as storage, caching, transactions, indexing, concurrency, gis support, backups, etc.

If anything, devs probably just need to be more familiar with DB internals and have tools setup to analyze queries easier.

Re:How and why the Relational Model works for databases by oblio on Monday January 17, 2022 @09:50AM (#29967047)
True, but you really, really want to be sure that if you're going the NoSQL route, your manual schema management will be top notch or your data structures will be super simple and the query demands will also be super simple.

Most of these high performance use cases actually fit this model well (well, the schema management being awesome is the wildly varying part).

But for everything else, there's Mastercard and RDBMS.

Re:How and why the Relational Model works for databases by cle on Monday January 17, 2022 @09:56AM (#29967121)
Yes, you should think about your domain and requirements, and the technical tradeoffs, before picking a DB technology. Don't just run with RDBMS b/c it has better theoretical modeling capabilities, your customers don't care about that.

For enterprise stuff with relatively low traffic and high amounts of complex ad-hoc queries, RDBMS is without a doubt the best choice. If you have high traffic web services with strict availability and latency requirements, I would seriously consider avoiding RDBMS's as they tend to be difficult to operate and scale with those requirements, and they let you easily do terrible things in that context (e.g. locking behavior, txid exhaustion, etc.).

Re:How and why the Relational Model works for databases by LAC-Tech on Monday January 17, 2022 @05:32AM (#29965190)
You're familiar with every NoSQL database?

Impressive!

Re:How and why the Relational Model works for databases by oblio on Monday January 17, 2022 @06:01AM (#29965349)
There you go, I've edited my previous comment to account for pedantry.
Re:How and why the Relational Model works for databases by LAC-Tech on Monday January 17, 2022 @04:58PM (#29972157)
I guess I should just accept the term that "NoSQL" now pretty much means "Monngo and Dynamo". Because it feels weird to group key-value stores, columnar stores and graph DBs under one term.
Re:How and why the Relational Model works for databases by oblio on Monday January 17, 2022 @05:43PM (#29972544)
Well, I don't want to be super mean to you or NoSQL, but NoSQL itself was a bit of a fad.

Columnar stores and graph DBs have always felt more like "real" tech, something promoted for specific use cases.

Key-value stores I'm not even going to be denigrating at all since we've had them since forever and they do a great job at their tasks (BerkeleyDB, Memcache, Redis, etc.). They don't deserve being included into fads or niches, they're a very valuable universal resource.

At least that's how I see it.

Re:How and why the Relational Model works for databases by LAC-Tech on Monday January 17, 2022 @07:14PM (#29973238)
Yeah but in my mind columnar stores and graph stores are NoSQL databases. Because they're not relational.

But the term has shifted underneath me. NoSQL means document database.

And - in practice - it means using a document database as the primary store when it's not really the right choice (been there!)