News for hackers. Stuff that matters.
Re:The Great Migration from MongoDB to PostgreSQL by onetimeuse92304 on Friday March 29, 2024 @05:21AM (#39862061)
Ah, the old "I used a wrong product for my problem and now I complain the product is bad because it does not suit my case." defence.

MongoDB is a document database. It is not supposed to be good at relations. Also lacking support with cloud providers and lacking experience with MongoDB is not MongoDB's problem, it is your poor decisionmaking. If you value those things, you should have taken it into account when you were choosing MongoDB in the first place.

Now, I am not a big lover of MongoDB. Some years ago I was forced to use it and I had very low opinion of it as quite immature product. It is still far behind in maturity compared to something like Oracle database or PostreSQL, but in the meantime I learned to appreciate some of the things MongoDB is good at.

I also admit that MongoDB's transactions are a total joke. It should be prominently placed in the documentation that you are using them at your own risk. I don't use MongoDB transactions anymore because there are better ways to architect your application with MongoDB without using transactions.

I like MongoDB for the ease of use when rapidly prototyping things. I like the concept of a no-schema, document database and some of the additional features it provides to deal with the document. I like its reactive java driver which is a breeze to use to construct data pipelines quickly. I like change streams.

In the end, I think it is good to have a selection of tools that are good at doing different things.

It is our responsibility to chose the right tool for the job. If you chose poorly, don't try to fault the tool for it.

Re:The Great Migration from MongoDB to PostgreSQL by dangtony98 on Friday March 29, 2024 @08:17AM (#39863138)
Author of the article here!

The article itself does not "complain the product is bad" but some other comments in this thread would certainly suggest so. Instead, the article says that the use-case evolved, states reasons for why MongoDB was no longer suitable (while at some point it was) and why PostgreSQL was chosen instead, and discusses the migration process involved in the transition and the results.

Regarding lacking support with cloud providers and lacking experience with MongoDB is indeed NOT MongoDB's problem. However, it is a problem for users of the platform that are trying to self-host it with MongoDB as a database dependency and therefore justifies the PostgreSQL migration with PostgreSQL being a better candidate for this use-case.

To be clear, the article is not saying that MongoDB is bad and no fingers are being pointed. I would in fact say that there are an array of use-cases where MongoDB is an excellent choice (e.g. I resonate with the point on rapid prototyping).

Please don't skew the words.

Re:The Great Migration from MongoDB to PostgreSQL by finaard on Friday March 29, 2024 @09:42AM (#39864027)
To me the article read more that PostgreSQL might have been the better choice from the beginning - but due to lack of experience in the team MongoDB was "good enough" at that point.
Re:The Great Migration from MongoDB to PostgreSQL by tkellogg on Friday March 29, 2024 @10:27AM (#39864514)
I recall when my team chose MongoDB ~2011, Postgres & friends didn't have JSON columns, so there was a lot of extra data modelling that probably was unnecessary.

The biggest use case for MongoDB was for huMongous data. Obvs MongoDB was a good fit, because of the name.

Re:The Great Migration from MongoDB to PostgreSQL by vbezhenar on Friday March 29, 2024 @12:16PM (#39865808)
I still didn't have a single use-case to use JSON columns yet. Sometimes I feel that JSON columns are made for people who don't want to properly design their domain model. Big mistake.
Re:The Great Migration from MongoDB to PostgreSQL by onetimeuse92304 on Friday March 29, 2024 @01:40PM (#39866866)
I think it is the same misguided discussion as strongly, staticall typed vs weakly typed languages.

(Mind I mean the discussion is misguided, not you)

The problem is that relational model is a one solution to certain problem of when you want to pay the price of enforcing the model of your data. Relational database is when you pay all the cost upfront.

With MongoDB I have other options.

It does not mean they are necessarily better options. But some of them are better in some situations, for example when rapid prototyping. Or when you need to preserve data in different formats as the application evolves.

Personally, if you can, you probably should model your data upfront. But it is good to have other options, sometimes.

Re:The Great Migration from MongoDB to PostgreSQL by abound on Friday March 29, 2024 @12:24PM (#39865898)
The main use case I find myself reaching for JSON/JSONB fields is unstructured map-like metadata, like KV tags on a resource, where the data model/domain has no expectation that any given field is set.
Re:The Great Migration from MongoDB to PostgreSQL by unclebucknasty on Friday March 29, 2024 @12:52PM (#39866238)
Interesting point, but I wonder if there's a little confirmation bias and/or circular logic there.

In other words that you don't see a use case for JSON columns because you don't think it's good DB design.

I know I have used them after decades of RDBMS work, and it feels like I'm "cheating" a bit. But they really do make sense for some use cases that can be harder to achieve with traditionally normalized RDBMS alone. For instance, for configuration options or scenarios where different objects may have a few different attributes, but are largely the same.

If you've ever used an entity-attribute-value data model or otherwise ended up with table proliferation or a bunch of nullable columns to accommodate the latter use case, you might appreciate the simplification JSON columns offer.

Re:The Great Migration from MongoDB to PostgreSQL by jerf on Friday March 29, 2024 @11:01AM (#39864895)
Yes, I think it's reasonable to say the NoSQL community drove the JSON columns in the relational databases. And while the JSON-ness of said columns is not necessarily the relevant bit to me, having the escape hatch of being able to just sort of splat out a "other" column has been something I've used several times, and it has relieved some non-trivial stress and strain in relational designs in my opinion. It's not even that hard to migrate something out of the JSON column if you want to raise it to a full column later, and JSON indices mean you may not even really need to.

I have a hard time justifying a document database in 2024 when I have 2024 Postgres; I don't know how large the window is where MongoDB will do something Postgres won't, but I know my systems are not in that window. (Assuming it even exists, which I wouldn't be surprised it doesn't, but for the sake of argument I will assume it does.) But a decade ago that would be a much harder decision if I truly had a document-based use case.

Re:The Great Migration from MongoDB to PostgreSQL by jimbokun on Friday March 29, 2024 @12:21PM (#39865867)
Horizontal scaling is still challenging in Postgres. That’s where “NoSQL” systems still have their place. Or one of the SQL systems layered on to a NoSQL architecture, with the trade offs that entails.
Re:The Great Migration from MongoDB to PostgreSQL by wlll on Friday March 29, 2024 @11:48AM (#39865489)
https://www.postgresql.org/about/featurematrix/#json

You're right, 9.4/9.5 were when JSONB was introduced and expanded (release 2013/2014).

That said, it's a big decision to go with two very very different technologies (relational vs Document store) just for a column type, storing JSON must have been a pretty major product feature?

Re:The Great Migration from MongoDB to PostgreSQL by lloydatkinson on Friday March 29, 2024 @10:40AM (#39864640)
I have noticed in recent times comments on here have been getting more and more like this. An article I posted a few weeks ago got the same treatment even though I iterated multiple times the same points as I anticipated these kinds of comments from people that didn’t read it fully.
Re:The Great Migration from MongoDB to PostgreSQL by thih9 on Friday March 29, 2024 @05:26AM (#39862087)
To be fair, Mongodb’s pr team did their best to present it as the solution to all problems.

> In 2012, 10gen’s VP of Corporate Strategy Matt Asay argued “there will remain a relatively small sphere of applications unsuitable for MongoDB … the majority of application software that developers write will be in use cases that are better fits for MongoDB and other NoSQL technology … Those functions that really help a company innovate and grow revenue [will be NoSQL].” He would note that we were living in the post-transactional future.

https://news.ycombinator.com/item?id=15124306

Re:The Great Migration from MongoDB to PostgreSQL by onetimeuse92304 on Friday March 29, 2024 @06:26AM (#39862434)
To be fair, almost all product PR teams do the same. I just ignore most of what they say in favour of my own opinion that I try to build based on reading actual technical documentation, experimenting with the product, etc.

Anybody who makes tech decisions based solely on what PR teams say is naive and incompetent at best.

Re:The Great Migration from MongoDB to PostgreSQL by benterix on Friday March 29, 2024 @09:33AM (#39863939)
> To be fair, almost all product PR teams do the same.

As it is their "duty" to twist the truth in this way, it is our role as the users of these technologies to present things as they are.

Re:The Great Migration from MongoDB to PostgreSQL by thih9 on Friday March 29, 2024 @10:42AM (#39864672)
> To be fair, almost all product PR teams do the same.

No, mongodb’s PR approach and results were largely uncommon at the time. They didn’t advertise directly but instead targeted and amplified dev communities - as detailed in the article linked earlier.

The result was that people not associated with mongodb talked about mongodb at various dev conferences and in blog posts. People didn’t want to listen to PR teams then too, but they followed their peers.

Re:The Great Migration from MongoDB to PostgreSQL by onetimeuse92304 on Friday March 29, 2024 @10:59AM (#39864871)
Again, I will stick to saying that if you use technology you have no experience with, the problems resulting from it are on you.

Testing out technology on important projects is reckless and is a risk a lot of people take but push away any responsibility for.

As a professional, before I suggest to use a technology at a project, at first at the very minimum I spend some time with it to learn the ins and outs. Every technology has cons. Just because you don't know them does not absolve you from responsibility for them.

The reality is that most applications are boringly similar once you define the problem in simple words. There is very rarely a need to use new technology. It is my experience that it is usually better to use technology that is not necessarily cutting edge but one that you know well and that you can easily hire people who know it well.

Because no technology is perfect, but it is much better to know the cons ahead of time and know you will have people who will know the problems and who can find reasonable solutions for.

I am personally sick and tired of people who first advocate for new technology to be added to the project, touting its supposed benefits, not knowing the cons, then don't want to take responsibility once we actually find the dead bodies.

Re:The Great Migration from MongoDB to PostgreSQL by rokkitmensch on Friday March 29, 2024 @11:22AM (#39865165)
The funny thing is that they chose Mongo precisely because they were familiar with it...
Re:The Great Migration from MongoDB to PostgreSQL by zzzeek on Friday March 29, 2024 @10:50AM (#39864756)
MongoDB came into the community like a ton of bricks, taking over conferences, flooding the zone with mugs and t shirts, and for a time there, you couldn't view the front page of hacker news without at least two MongoDB posts. The death of SQL, now known to be an inferior relic of the past, was a regular topic of discussion. There's a reason why the "Web Scale" meme is so famous, because that's what actually happened for awhile there. SQL was an inferior, leaky abstraction, and ACID was not "web scale". it became all about the CAP theorem (which I hardly ever see anyone writing about these days).

only a year or two later, when teams that went all in on MongoDB at the behest of MongoDB's marketing department started realizing they'd been sold a bill of goods, did the slow and arduous march back to what continued to be the best generalized solution for 95% of data problems, the ACID compliant relational database, begin to occur. given that, this blog post seems really behind the times.

Re:The Great Migration from MongoDB to PostgreSQL by swasheck on Friday March 29, 2024 @11:40AM (#39865402)
mongo was absolutely one of the most egregious (daresay dishonest?) purveyors of such myth. it was quite off-putting for me and my team.
Re:The Great Migration from MongoDB to PostgreSQL by taffer on Friday March 29, 2024 @12:09PM (#39865723)
As a counterexample, Azure Cosmos or Amazon DynamoDb are marketed as specific solutions for specific problems. Other RDBMS like SqlServer or Postgres are marketed as general purpose solutions because they are in fact general purpose solutions. Mongodb, on the other hand, is a specialized "document database" marketed as a replacement for relational databases.
Re:The Great Migration from MongoDB to PostgreSQL by gwbas1c on Friday March 29, 2024 @10:58AM (#39864848)
They were doing that back in 2010. I went to a few conferences where they promoted MongoDB as an all-purpose database that was the "next thing" compared to SQL.

Once everyone realized its shortcomings as a general-purpose database, there was a gradual "oh shit this isn't working" migration back to SQL-based databases.

I think the big problem is that the MongoDB programming model and APIs are very nice; we really need a database that is still relational "under the hood" but has an API that's more like MongoDB.

Re:The Great Migration from MongoDB to PostgreSQL by bashinator on Friday March 29, 2024 @12:01PM (#39865632)
I believe one of the job interview problems at Mongo for software engineers, was to design a relational layer on top of the document store.
Re:The Great Migration from MongoDB to PostgreSQL by liquidgecka on Friday March 29, 2024 @12:18PM (#39865825)
In 2010-2011 time frame the mongo team sponsored an effort inside of Twitter to replace MySQL with mongo. There was a group of us that worked to migrate the tweet store to Cassandra that were able to talk with leadership and get that initiative killed. Turns out migrating HIGHLY structured data into mongo was never a good idea, especially at that scale.
Re:The Great Migration from MongoDB to PostgreSQL by CuriouslyC on Friday March 29, 2024 @07:46AM (#39862883)
If mongo is just a document database, there's really no reason to use it over elastic. The query story is slightly nicer with mongo, but we're not doing relational algebra here, right? Elastic crushes mongo at literally everything else.
Re:The Great Migration from MongoDB to PostgreSQL by finikytou on Friday March 29, 2024 @10:17AM (#39864386)
i guess elastic is more heavy setup for some POC.
Re:The Great Migration from MongoDB to PostgreSQL by CuriouslyC on Friday March 29, 2024 @05:59PM (#39869449)
That might have been true in the past but it's not hard to PoC with elastic using docker compose.
Re:The Great Migration from MongoDB to PostgreSQL by synthc on Friday March 29, 2024 @08:38PM (#39870637)
Elastic is much more of a pita to maintain and monitor than Mongo
Re:The Great Migration from MongoDB to PostgreSQL by LtWorf on Friday March 29, 2024 @10:36AM (#39864603)
ES had a bug where a syntax error in a query would send it in a weird state that would then give wrong answers to all queries until the process got rebooted. Which of course isn't fast.

I reported the issue, and thankfully I changed job and never had to deal with ES again. After a few years they contacted me to ask if the issue was still there, I said I had no idea.

I have no idea if the issue is still there.

Re:The Great Migration from MongoDB to PostgreSQL by pydry on Friday March 29, 2024 @07:27AM (#39862761)
The issue isn't that mongodb isn't suitable for some problems. I've used it before and it has not caused issues. It's that the problems it is suitable for are a subset of those that postgres is suitable for.

I can just as easily rapidly prototype in postgres. If I want to store schemaless JSON in postgres I can easily do that.

>In the end, I think it is good to have a selection of tools that are good at doing different things.

In the end I think every tool needs a niche - something it is good at that its competing tools are not good at.

Mongo doesnt have that.

Re:The Great Migration from MongoDB to PostgreSQL by JJMcJ on Friday March 29, 2024 @09:19AM (#39863762)
> MongoDB is a document database.

Thank you. A one sentence clarification of the issues in Mongo vs RDBMS.

Re:The Great Migration from MongoDB to PostgreSQL by benterix on Friday March 29, 2024 @09:37AM (#39863977)
It's a bit more nuanced than that. PostgreSQL supports JSON/JSONB data types so if you need, you can use it in a way similar to MongoDB, with less structure and the remaining caveats, but with transactions and all other goodies working out of the box and being able to use standard RDBMS features if needed.
Re:The Great Migration from MongoDB to PostgreSQL by throw_m239339 on Friday March 29, 2024 @11:01AM (#39864905)
> MongoDB is a document database.

So can PostgreSQL be with JSON types and queries.

Re:The Great Migration from MongoDB to PostgreSQL by 999900000999 on Friday March 29, 2024 @11:32AM (#39865302)
>As part of that stack, we chose MongoDB + Mongoose ORM because the combination presented least overhead and allowed us to ship quality features quickly. As Sir Tony Hoare states, “premature optimization is the root of all evil,” and there was certainly no need for further optimization at the time.

Looks like the tool did a great job while they were getting started. Mongo is very easy to hack something together with, later on it looks like they just needed to migrate to something more stable

Re:The Great Migration from MongoDB to PostgreSQL by swasheck on Friday March 29, 2024 @11:38AM (#39865370)
yeah ... in the early days of mongo it was pitched as a replacement for relational because a) it had faster writes without those pesky joins and constraints, and b) because it's too hard to model data, so chances are it's just unstructured. the onslaught of "relational is dead" and "joins are annoying" was tremendous
Re:The Great Migration from MongoDB to PostgreSQL by 999900000999 on Friday March 29, 2024 @02:59PM (#39867701)
Plus there's a good chance they didn't know exactly what they needed to build when starting out.
Re:The Great Migration from MongoDB to PostgreSQL by forgotmyinfo on Friday March 29, 2024 @11:45AM (#39865456)
Nonsense. Choosing the correct database isn't "premature optimization", it's the bare minimum of being a competent programmer. Why do we never have time to do it right, but we always have time to do it over?
Re:The Great Migration from MongoDB to PostgreSQL by 999900000999 on Friday March 29, 2024 @12:09PM (#39865720)
It's generally faster to get started with Mongo.

If you're talking about a startup, you might not even get to the let's do it over stage. You might have six months to just hack something out so you can raise more funding, and in that case Mongo is a great choice.

The needs of your business might also just evolve over time.

Re:The Great Migration from MongoDB to PostgreSQL by jimbokun on Friday March 29, 2024 @01:52PM (#39866968)
Postgres and MySQL are easy to hack something together with, too.
Re:The Great Migration from MongoDB to PostgreSQL by 999900000999 on Friday March 29, 2024 @02:17PM (#39867262)
That's assuming you're experienced with them, if you know next to nothing about databases, Mongo is a much better choice. In this case it looks like when they got started the team just didn't have all that much experience with databases.
Re:The Great Migration from MongoDB to PostgreSQL by forgotmyinfo on Friday March 29, 2024 @11:44AM (#39865441)
Sorry, 100 times out of 100, "rapid prototyping" means "we built this with the wrong database, now we're stuck with it". If it isn't obvious how to store your data up front, then you either aren't planning your software well, or you don't actually know what you're building.
Re:The Great Migration from MongoDB to PostgreSQL by wlll on Friday March 29, 2024 @11:50AM (#39865508)
I was also a bit confused by that part. What is it about Postgres that prevents rapid prototyping? It's not like the schema is set in stone, or even particularly hard to change once you've created it.
Re:The Great Migration from MongoDB to PostgreSQL by benterix on Friday March 29, 2024 @12:00PM (#39865621)
Well, yes and no. If you already have your database running in production with many transactions and live connections 24/7, changes to the schema might not be hard per se but always needs careful planning and execution. Additionally, in Postgres a change like adding a column locks the whole table (although I hear this is going to change) so writes are off for a short time. If this becomes unacceptable, you go for a blue/green deployment which has its own gotchas.

So, while I agree with your main point - there is nothing in Postgres that prevents rapid prototyping, and I would chose it over Mongo any moment, I understand why some people might prefer the more "dirty" approach.

Re:The Great Migration from MongoDB to PostgreSQL by wlll on Friday March 29, 2024 @12:09PM (#39865718)
True enough, and the solutions for lighter-weight schema changes have evolved and weren't always so good, but early stage startups often don't really have the sort of data-weight issues that make this hard, unless they're starting with large data sets already. Even at the scale of small hundreds of millions of rows per table (like the company I'm in charge of the database for) it's not much of an issue.

I had to look it up, apparently adding a column with a non-null default was "fixed" in PG 11 (2018), but with a null default it had been fast for a while:

https://www.depesz.com/2018/04/04/waiting-for-postgresql-11-...

Re:The Great Migration from MongoDB to PostgreSQL by teaearlgraycold on Friday March 29, 2024 @12:00PM (#39865618)
> I like MongoDB for the ease of use when rapidly prototyping things.

I think this is similar to when people say writing tests speed them up through TDD, then others don’t write any tests at all. What I take away is you can do it either way as long as you’re bought in and are familiar with the process.

For me it’s no trouble at all to define my schema in advanced. And Postgres provides me with the json column escape hatch. Combined with the massive benefit of the data validity guarantees later in the app’s lifecycle I pick Postgres every time.

Re:The Great Migration from MongoDB to PostgreSQL by taffer on Friday March 29, 2024 @12:01PM (#39865644)
So what is a good use case for Mongodb? I have never seen an application where a "document database" would have been a good choice. In every project where I have seen Mongodb, it has turned out to be the wrong choice.

Even for prototyping there are many other good choices since RDBMS like Postgres have implemented JSON support. Mongodb looks like a solution in search of a problem.

Re:The Great Migration from MongoDB to PostgreSQL by joneholland on Friday March 29, 2024 @12:05PM (#39865683)
Applications that are essentially form wizards are a great fit for a document database.

Think application forms etc.

Re:The Great Migration from MongoDB to PostgreSQL by wlll on Friday March 29, 2024 @12:39PM (#39866066)
Is there a benefit of Mongo for that over just Postgres with JSONB columns? You're still storing JSON in Postgres but you'd get the relational aspects too for things like users having many forms, billing and account relationships etc.
Re:The Great Migration from MongoDB to PostgreSQL by joneholland on Friday March 29, 2024 @12:56PM (#39866285)
Easier horizontal scaling and organizational inertia would be the main reasons to use mongo over a jsonb column. I wouldn’t introduce it to a psql shop if they are already great at running psql.
Re:The Great Migration from MongoDB to PostgreSQL by unclebucknasty on Friday March 29, 2024 @01:07PM (#39866413)
No benefits that I can think of at the database layer. Postgres's addition of JSONB columns represents the best of both worlds. Funny to think that all of that noise about nosql replacing rdbms was essentially nullified by the addition of one column type.

Some people do like the MongoDB API, however.

Re:The Great Migration from MongoDB to PostgreSQL by jeff-davis on Friday March 29, 2024 @12:37PM (#39866051)
"It is our responsibility to chose the right tool for the job."

That perspective doesn't work well for database products, in my opinion. There is a huge pressure for databases to evolve with your business and applications and to adapt to whatever your throw at it.

Swapping out a database product is less like changing tools and more like changing a foundation. You can't do it every time a new problem arises.

That's not to say you can't use a few different products if that makes sense. But that has its complications.

Re:The Great Migration from MongoDB to PostgreSQL by unclebucknasty on Friday March 29, 2024 @12:42PM (#39866096)
To be fair, it burst onto the scene back in the "all you need is NoSQL...RDBMS is dead" era.

So it became one of those software dogma things and I remember having "debates" right here on HN, including around the necessity of transactions.

So it's kind of ironic to see people now chastising those who fell prey to the dogma. Full circle.