Re:MongoDB’s New Query Engine
by coldtea on Thursday September 21, 2023 @06:48AM
(#37595822)
I thought all those points died along with the hype ("MongoDB is web scale"), as all were wrong.
Those were the marketing points people mentioned in their "honeymoon phase" posts. Then after using it in production, actual benchmarks and comparisons coming in, came the regret and moving on posts. In fact, most of those mentioned moving from Mongo to Postgres, and there was a full blow "yeah, NoSQL was a dumb idea for 99% of use cases, and Mongo even more so" discussion.
In the end MongoDB was the butt of a joke, there were whole memes about it.
So this comment is like a trip down memory lane, or into an alternate universe, where it's like 2012 and these things never happened.
Re:MongoDB’s New Query Engine
by threeseed on Thursday September 21, 2023 @06:53AM
(#37595854)
At least in my experience it is all still the case today.
If you are dealing exclusively with documents I find MongoDB to be faster, better and easier to use. If my data model is hybrid or purely relational then I would use PostgreSQL.
After all these years I am still waiting for an horizontal scalability solution for PostgreSQL that is up to the level of a modern database. It's 2023 and for many of us in enterprise environments it's a mandatory NFR.
But it seems like you're not interested in a having an actual technical discussion.
Re:MongoDB’s New Query Engine
by aeyes on Thursday September 21, 2023 @04:31PM
(#37603884)
It has serious limitations and needs a lot of elbow grease to run at scale. With some of the alternatives (for example MongoDB) you basically just add a few more machines and you are good.
In my opinion Citus is still only a hack which is only useful if for some reason you can't move away from Postgres but you have reached the limit of what you can get out of a single machine.
Re:MongoDB’s New Query Engine
by threeseed on Thursday September 21, 2023 @08:40PM
(#37606446)
There are many examples but let's take one: star schemas.
I can implement that in MongoDB such that a query can be done in O(1) whereas implementing this in a relational database requires n joins which in turn requires n foreign key indexes and n row fetches. In many cases this approaches O(n). I have personally seen improvements of 10-100x for this use case when n is large.
Also there is a reason that the standard now in enterprise companies are data lakes and not EDWs. Because the cost and effort to model all of your data is prohibitive especially when 99% of the data ends up never being queried.
Re:MongoDB’s New Query Engine
by coldtea on Thursday September 21, 2023 @06:59AM
(#37595884)
Between plain old horizontal scaling via custom sharding, replicas, and extensions like Citus and Timescale that offer full horizontal scalability, Postgres is handling some of the biggest use cases in the world.
Re:MongoDB’s New Query Engine
by threeseed on Thursday September 21, 2023 @07:06AM
(#37595938)
Citus was acquired by Microsoft so there are doubts about its future longevity given that so many similar promises of ongoing support are rarely maintained. Timescale is optimised for time-series so not sure it's applicable.
Point still remains that PostgreSQL lacks a modern, built-in horizontal scalability solution.
Re:MongoDB’s New Query Engine
by hot_gril on Monday September 25, 2023 @08:17PM
(#37652621)
Well let's see how long Google's Spanner lasts. It's designed to support huge-scale systems, and personally I'd be terrified locking a huge system into that.
Re:MongoDB’s New Query Engine
by hot_gril on Monday September 25, 2023 @08:16PM
(#37652616)
And it comes with a bunch of caveats. It's bolted-on horizontal scaling, vs MongoDB where that's the main feature. Yes I still prefer Postgres in general, but it's fair to point out that weakness of it.
Re:MongoDB’s New Query Engine
by maxbond on Thursday September 21, 2023 @12:03PM
(#37599658)
Fair, but this is getting fixed. Postgres 16 (which was just released) adds an origin tag to logical replication, so that you can know whether or not to pass on a change (and avoid having an infinite loop where a change is replicated back and forth). That's the first step to having multiple mains. I think in theory, if you had deep Postgres knowledge, you could set up a high touch, bespoke multi-main deployment today.
I expect the replication story with Postgres will get much better over the next few years.
Re:MongoDB’s New Query Engine
by LtWorf on Thursday September 21, 2023 @07:50AM
(#37596269)
They used it at my first full time job out of university.
It was terrible.
It was not fast, a regular SQL db would have been much better, due to bugs in our code, the data in mongodb was not uniform, which led to failures in the code when the document didn't have the expected format.
Cherry on top is that they were mostly growing documents, which is a very slow operation, that would have been done very fast with a many-to-many table in SQL instead.
Re:MongoDB’s New Query Engine
by skywhopper on Thursday September 21, 2023 @06:11PM
(#37605123)
It all depends on your data structure and problem space. It was silly to dismiss RDBMSes as obsolete 10 years ago, and it's silly to dismiss document databases or other NoSQL databases today. There are plenty of use cases for which Mongo is far superior to Postgres. A wise engineer chooses the best tool for the job.
Re:MongoDB’s New Query Engine
by jameshart on Thursday September 21, 2023 @07:24PM
(#37605830)
If you base your technology decisions on what the current memes are on /r/technology, you're going to have that impression, sure. But in reality, no, there wasn't a vote held in 2016 where we all agreed NoSQL was a mistake and we should go back to RDBMSs.
The horizontal replication story in Mongo is just a different game. And because it's so reliable, that ends up being how you solve not just availability and scalability but also engine upgrades and config changes.
I migrated a bunch of production transactional databases on Mongo Atlas from a GCP cluster to an AWS cluster, with zero read/write downtime, by just walking through simple cluster reconfiguration steps. Add some nodes in AWS... move the write master... remove some nodes from GCP. It was stress free and easy.
The same trick with replication on Postgres was a nail biter every time we did it.
Re:MongoDB’s New Query Engine
by riku_iki on Friday September 22, 2023 @02:40AM
(#37608662)
also jepsen report says that it is mongo marketing assessment, and they are not really acid:
MongoDB’s claim of “full ACID transactions” which “maintain the same data integrity guarantees you are used to in traditional databases” could be misleading. We recommend that users who are accustomed to serializable behavior evaluate critical transactions carefully, to identify whether running at snapshot isolation could violate application-level constraints. MongoDB may wish to revise their marketing language to use “snapshot isolated” instead of “ACID”.
https://jepsen.io/analyses/mongodb-4.2.6
Re:MongoDB’s New Query Engine
by varelaz on Thursday September 21, 2023 @05:24PM
(#37604582)
PostgreSQL will be horizontaly scaling if you can avoid joins and index range locks. But the thing is you don't want to, or why do you need SQL database then. I found MongoDB good for cases when you need a lot of upserts, your data model is looks like document and you don't need joins. Like you store events data for further processing or collect stats or counters with tags/indexes.
Re:MongoDB’s New Query Engine
by salil999 on Thursday September 21, 2023 @07:40AM
(#37596176)
This is completely false and has nothing to do with the article. DynamoDB and MongoDB might both serve JSON but they're used for wildly different things. DynamoDB is a key value store whereas MongoDB is much more than that.
Re:MongoDB’s New Query Engine
by JackMorgan on Thursday September 21, 2023 @06:47AM
(#37595816)
My question exactly. No sense building a house on someone else's land. Too many projects I've worked on were shattered because of technology that stopped being offered or got too expensive to justify.
Re:MongoDB’s New Query Engine
by nemo44x on Thursday September 21, 2023 @07:06AM
(#37595936)
Because just about no one cares and I want to burn down a backlog quickly and often mongo enables that. Developer productivity is far more valuable that open source purity.
Re:MongoDB’s New Query Engine
by KronisLV on Thursday September 21, 2023 @07:18AM
(#37596009)
> Developer productivity is far more valuable that open source purity.
I mean, at least you get access to MongoDB source code, so that is something.
I remember working on a project that used Clusterpoint years ago and the problem was that it was basically an abandoned piece of software due to the company behind it no longer releasing new versions. It was running, yes, but getting data out of it and migrating to something else sure took work and was very annoying because a lot of functionality had been built around it.
So in regards to something so critical as databases and data storage solutions, I'd err on the side of picking things that are open source, or have compatible alternatives with permissive licenses.
If someone were to build their own business around MongoDB (say, offering it as a service) and then SSPL came along, they'd be done for, even if just because of getting caught up in things going on between the org and large cloud vendors or something. Same with how you have to be careful with AGPL, so something like MinIO might be a non-starter in some cases.
Re:MongoDB’s New Query Engine
by KronisLV on Thursday September 21, 2023 @07:58AM
(#37596337)
> Mongo is a thriving database so I’m not worried about it being abandon.
With this, I can totally agree! And as long as it's not AGPL, you should be fine using it for most regular development stuff, even for closed source projects.
Re:MongoDB’s New Query Engine
by nemo44x on Thursday September 21, 2023 @10:05PM
(#37607030)
Yeah sure I guess but that’s not a real problem, at least not now or in the near future. Good chance it won’t even be a project I need to budget for by then.
I’m really more interested in getting shit done today rather than worrying about 100000 possible future cases. This is how you succeed.
Re:MongoDB’s New Query Engine
by nemo44x on Thursday September 21, 2023 @07:03AM
(#37595921)
Because the developer productivity is great because the language drivers that interface with the database are seamless with your code. MongoDB lets you burn down a backlog better than any other database. It’s not right for every problem but when it is right it’s the best tool because the language drivers are so good. That’s their secret sauce.
Re:MongoDB’s New Query Engine
by dubcanada on Thursday September 21, 2023 @07:26AM
(#37596060)
Why is there no conversation on the topic at hand and instead conversation on why use MongoDB.
This authors post has nothing to do with pro/cons of mongodb, and is entirely around a new query engine and computer science. And instead we've devolved into a discussion on the pros and cons of PostgreSQL vs MongoDB.
Re:MongoDB’s New Query Engine
by christophilus on Thursday September 21, 2023 @07:35AM
(#37596118)
Same as it ever was. Someone posts a new JS framework, and the top 50 comments are about how bad JS is, and why don't we do RUST with WASM instead? Someone posts about a new Go release, and the top 50 comments are about how terrible Go is, and why don't we use a real language instead? Mongo... same thing.
I get it. People don't like certain tools. But it would be nice for the rest of us to be able to actually discuss the topic at hand without all of the tangential critical noise.
Re:MongoDB’s New Query Engine
by sambeau on Thursday September 21, 2023 @07:57AM
(#37596330)
It's completely understandable. This is a significant and interesting upgrade to MongoDB that naturally gets people interested in a, "Hmmm… maybe I should take another look at MongoDB" way. Before wasting time, it seems sensible to use this forum full of experts to ask whether it would be worthwhile, and what the advantages or disadvantages would be. I didn't sense any negativity in the question, and it was the exact question I had in my head too. After skim-reading the article I went straight to scrolling down the comments to look for this exact question.
Re:MongoDB’s New Query Engine
by h1fra on Thursday September 21, 2023 @07:29AM
(#37596084)
I'm a postgres fan but I have to admit the JSON syntax is not the best and not well supported by all the tooling around this database. Doesn't mean it's bad but the DX could be better.
Other than that, I have never seen a proper justification to use schema less db, good for prototyping and get started, not incredible as a long-term solution.
Re:MongoDB’s New Query Engine
by synthmeat on Thursday September 21, 2023 @07:46AM
(#37596232)
Schema-less is a bit wrong term, since you always end up with one but, indeed, schema-on-read vs schema-on-write discussion for individual use cases is far from a settled thing, even though the zeitgeist is that schema-on-write won sometime in the 70s.
You say "prototyping", but I would generalize that to "faster evolving" in the long term. Of course, not without tradeoffs.
I personally have zero issues writing a custom marshaller when needed for any the schema-less document collections I have. Constraints your application has on the data are a superset of db schema anyways.
Re:MongoDB’s New Query Engine
by synthmeat on Thursday September 21, 2023 @07:33AM
(#37596107)
Just to add to sibling comments, one killer feature for me is ChangeStreams[1]. It's miles ahead of what Postgres[2] offers, and it enables really interesting use cases. Some of my services built around ChangeStreams end up not doing a single query to the DB. Data is right there in the program memory, indexed how I need it to be, and program is immediately reactive to any changes in the DB.
Re:MongoDB’s New Query Engine
by heipei on Thursday September 21, 2023 @04:24PM
(#37603772)
Serious counter-question: How do you run Postgres in a native and hands-off replicated failover setup? With MongoDB you create a three-node replica set and you're done. I've yet to see a simple guide for Postgres. But I might be wrong, so happy to hear how you would achieve the same with Postgres.
Re:MongoDB’s New Query Engine
by Too on Friday September 22, 2023 @02:40AM
(#37608664)
This isn't just a server property. It need to be implemented across the whole vertical, including client libraries and how queries are constructed in the application. With mongo it all is, and it's there out of the box.
Client libraries are made to do this failover and retry under the hood. Queries are more often made as upserts instead of assuming transactions, eg ObjectId rather than auto increment and so on. Even non upserting operations like $inc are safely retried if handled by queries and not application code.
It's magic to see it happen, with no extra code, the application just keeps running even when one server is down. Starting it up again and it reconciles with no hands-on.
I would love to see a postgres that does this, because relational data is often more useful than the document model.
Re:MongoDB’s New Query Engine
by masukomi on Thursday September 21, 2023 @07:31AM
(#37596099)
Dear Geeks with interesting things to say on your blog:
Please, for sake of the people you want to hear your words. DO NOT USE A SANS SERIF MONOSPACED FONT FOR ANYTHING BUT CODE.
I get it. We stare at text (code) formatted in this kind of font all day long, and many of us find fonts that we truly enjoy. But, most of our monospace sans-serif fonts are designed to make sure individual characters aren't misread. We don't read prose the same way we read code. There is far more pattern recognition going on than actual parsing of individual letters, and monospaced fonts break that. We can debate the aesthetics of serifs but they actually do help provide context clues to the pattern recognition systems in our brain.
Convincing you all to start using serif fonts on prose is not a battle i'm likely to win, but maybe I can convince you to only use monospaced fonts for your terminal, and your code.
Re:MongoDB’s New Query Engine
by weinzierl on Thursday September 21, 2023 @08:09AM
(#37596445)
You exaggerate the issue. Proportional fonts were a necessity when lead, ink and paper were expensive. In lead typesetting they are easy to do. This caused the success of newspaper fonts like Times New Roman.
Contemporary fonts have much less variance in the width of characters, except for a couple of outliers like the i. From
there to a completely monospaced font is not as big a leap as you make it seem. For me, I'm fine with monospaced fonts for prose.
Re:MongoDB’s New Query Engine
by TylerE on Thursday September 21, 2023 @12:11PM
(#37599792)
Disagree. Reading monospaced prose sucks big time. For those of us with a little bit of vision deficiency the vary character widths are helpful.
Also, the reduced character width (as a monospaced font inevitable has to be spaced at what the widest chatacters, such as w, require, means that you have to use a smaller font size to get the same amount of info per scroll/line/page whatever, again compromising readability.
Here's a comparison between the posted site, and the same site with the font size bumped up 4px, proportional (just browser default, I didn't cherry pick) and the ridiculously wide line spacing reduced.
Even with all that, the easier to read version is quite a bit more compact. Could likely bump the font size 2 more px and still be smaller.
Re:MongoDB’s New Query Engine
by ltbarcly3 on Thursday September 21, 2023 @11:31AM
(#37599176)
I'm surprised when people notice fonts. If the font is weird, or very decorative then sure, I would notice a font in Papyrus or Comic Sans. On my kindle I set it to the modern sans sarif font because its easier to read, for me. If there was no option to change the font I probably wouldn't have noticed, though.
I don't want say this in a way that comes off as insulting, but I'll just say it and please don't take it as a put down: if my kid came to me and expressed this much frustration and difficulty because of a sans sarif or monospaced font I would be concerned they had a problem with their vision or an issue processing what they saw, and I might research and/or take them to get checked out.
Re:MongoDB’s New Query Engine
by wheels on Thursday September 21, 2023 @11:36AM
(#37599257)
I wouldn't be bothered about it in this case (I'm much more offended when the font size or margins are stupid), but to use your phrase "I don't want say this in a way that comes off as insulting, but I'll just say it and please don't take it as a put down..."
You probably just don't have a strong sense of aesthetics. Some people notice design-y stuff and some people don't. A lot of people do notice fonts, or misaligned margins, or whatever. And a lot of people don't. The latter category are probably never going to be good designers or artists or whatever. People are different.
Re:MongoDB’s New Query Engine
by ltbarcly3 on Thursday September 21, 2023 @05:21PM
(#37604547)
You are right, while I might notice that sort of thing you mention it wouldn't distract me much. There are other things that completely make it impossible for me to function, like direct lighting in or near my field of view. I could claim like you do that people who have a bright light bulb hanging right next to their tv are just blind to some deeper experience, but probably they just aren't bothered by it.
What makes you think these proclivities about font choice are universal then? Maybe you are just insisting on what you find aesthetically appealing, while many people would equally validly feel the opposite?
Re:MongoDB’s New Query Engine
by wheels on Friday September 22, 2023 @04:08AM
(#37609287)
Fixed width fonts have a special purpose. They're designed to fill that purpose, not to be optimal for other things.
There's an age-old debate of how much "good taste" is subjective, but in this particular case, it's someone hammering a nail in with a wrench. Some will be bothered, others not, but it's definitely the wrong tool for the job.
The example of not liking lights in certain places isn't a very good one. It's an attempt to elide all "things people don't like" into subjectivity, but there are actually things that are better and worse. You would agree that website design can be better or worse, right? And that some people would have a better eye for good or bad design? With me so far? I'm saying you're probably not one of the people out on the end of the spectrum that's best able to recognize good and bad in design, but I hope you recognize that good and bad design exist, and that people vary in their ability to recognize it.
Re:MongoDB’s New Query Engine
by ltbarcly3 on Friday September 22, 2023 @11:20PM
(#37620329)
I think the inference that goes like 'if I can recognize something, there must be a corresponding faculty for that ability to recognize' is so remarkably bad that it is called a 'howler' in philosophy classes.
Well lets consider it anyway. Lets say you really are just the absolute best at having an aesthetic sense. What is aesthetics? It's knowing what looks good. What does it mean to look good? I means other people think it looks good too. Ipso facto, having a sense of aesthetics must be extremely common, since whatever the largest group of people agree looks good is what 'looks good' to people. So congratulations on your very common ability I guess.
Or maybe you are more of a snob, and your aesthetic sense is so rarified other people can't even begin to appreciate the things you think look good or bad. Lots of things you think are very beautiful would be called ugly by the vast majority of people, and what you think is ugly might generally be considered beautiful. Nobody understands other than the other elite aesthetes, which you recognize because they agree with your judgements. I could say something snide about how everyone has some group of people that agree with their judgements, and all that makes one group or another elite is that they say they are, but I'll just say congratulations, and point you in the direction of a modern art museum.
Re:MongoDB’s New Query Engine
by ltbarcly3 on Saturday September 23, 2023 @02:40AM
(#37621096)
So you are saying some website designs can be better than others? And you can tell which is which, but I can't? I've never seen someone try to appeal to their own innate sense of style as an authority before, but it's been neat to see.
Not to make it personal, but if you are going to claim some kind of innate design ability, you should remove the link to your website from your profile.
Re:MongoDB’s New Query Engine
by ltbarcly3 on Saturday September 23, 2023 @02:43AM
(#37621111)
I want to say to you, in earnest, things will get better and a bad period in your life will almost always unexpectedly improve sooner than you would think.
Re:MongoDB’s New Query Engine
by wheels on Sunday September 24, 2023 @03:41PM
(#37636481)
This an extremely weird passive aggressive flex. The thread basically went, "I think people that notice font problems have something wrong with them." To which I respond, "No, like designers and designy people notice stuff like that." And then to go from there to this is pretty, bwah?
Re:MongoDB’s New Query Engine
by ltbarcly3 on Monday September 25, 2023 @11:40AM
(#37645635)
I wrote like 5 different replies, and in the end I came to the conclusion you are probably just unhappy more generally which is leading to this weird outlet for it. If I'm wrong thats fine, but I don't want to play with you anymore.
Re:MongoDB’s New Query Engine
by salil999 on Thursday September 21, 2023 @07:37AM
(#37596142)
This is great and all but I'm curious on the performance improvements. I'm surprised there are no graphs or charts when show improvement in latency, CPU usage, disk reads, etc
Re:MongoDB’s New Query Engine
by endisneigh on Thursday September 21, 2023 @07:51AM
(#37596281)
> Our most recent report on MongoDB 3.6.4 focused on causal consistency and linearizability in sharded collections. We found that sharded clusters appeared to offer linearizable reads, writes, and compare-and-set operations against single documents, so long as users ran with read concern linearizable and write concern majority. However, any weaker level of write concern resulted in the loss of committed writes. MongoDB’s default level of write concern was (and remains) acknowledgement by a single node, which means MongoDB may lose data by default. Although the write concern documentation does not make this clear, the rollback documentation states:
Re:MongoDB’s New Query Engine
by beembeem on Thursday September 21, 2023 @12:26PM
(#37600010)
This was prior to the introduction of transactions years ago. The versions stated in your comment are past EOL and not offered in Atlas today. Not relevant to today's reality.
Re:MongoDB’s New Query Engine
by nevi-me on Thursday September 21, 2023 @07:52AM
(#37596286)
Heh, I missed that there's a new query engine.
From [0] it looks like it was available from 5.1, also interesting is that you can't choose which engine to use, so I suppose it only works on subsets or queries that meet certain conditions.
This isn't the first time I hear something to the effect of "LLVM JIT is great, but it introduces a lot of query latency". I wonder if there are other JIT engines more suitable for compiling potentially small/simple queries.
Re:MongoDB’s New Query Engine
by jandrewrogers on Thursday September 21, 2023 @11:38AM
(#37599280)
LLVM has many advantages for JIT but it does not prioritize very low latency, choosing to optimize other properties instead. Consequently, LLVM tends to be more popular for systems targeted at analytical workloads where queries commonly have intrinsically high overhead latency. The latency isn't terrible but it is noticeable if you are running a low-latency "fast-twitch" workload.
There are other specialized JIT compilers that are essentially purpose-built to provide very low latency; for systems like MongoDB which are rarely used for serious analytical processing, you'd probably want to use one of these instead.
Re:MongoDB’s New Query Engine
by maxbond on Thursday September 21, 2023 @12:34PM
(#37600143)
I'm sure it can[1], with a somewhat harsh limit of 32 per batch, do you mean can you get documents in insertion order? The default _id is a random right-leaning identifier[2], so sorting by it would only get you an approximate insertion order (except in a deployment with a single server). However as the linked documentation demonstrates you could supply a strictly incrementing ID, if your architecture abides.
Re:MongoDB’s New Query Engine
by winrid on Thursday September 21, 2023 @02:14PM
(#37601634)
no, that means you can sort on a max of 32 fields per sort request, like fieldOne, fieldTwo, fieldThree...
There's no limit on number of sortable documents. Use an index to sort and it'll also be fast and use no extra memory. If you want to do an in memory sort there are memory limits, but you can also tell it to overflow to disk.
Re:MongoDB’s New Query Engine
by bit_flipper on Thursday September 21, 2023 @11:37AM
(#37599272)
This article seems to have inspired others to look at MongoDB again, so I'll give my thoughts after using it recently.
MongoDB Atlas is a surprisingly good managed database product. I'm not a huge fan of someone else running my databases, but I think it might be the best one you can run across any cloud. If you like MongoDB (and, ignore the memes, there is a lot to like nowadays), and are OK paying a bit more to have someone run your database, I'd strongly consider Atlas.
Re:MongoDB’s New Query Engine
by winrid on Thursday September 21, 2023 @02:17PM
(#37601681)
The problem is when you grow. They can be really tough to work with on pricing. Also, their licensing does not allow servers past a certain size. Can you imagine Oracle telling the CIA they can't use servers with more than 256gb of ram? Just silly.
Re:MongoDB’s New Query Engine
by bit_flipper on Thursday September 21, 2023 @03:44PM
(#37603133)
I'm not sure what experience you have, but I've run both their Enterprise licensed database on prem as well as migrated to Atlas and there have never been any licensing issues preventing vertical scaling of databases. One of our clusters on Atlas right now has machines larger than 256GB of RAM -- you're more limited by what your cloud vendor has available than Atlas.
Re:MongoDB’s New Query Engine
by winrid on Thursday September 21, 2023 @04:58PM
(#37604236)
Actually yeah for Atlas, I guess they automatically bill you as if it was 2-3x Enterprise Advanced licenses so there's no discussion. I thought it was the same as Enterprise Advanced but I guess not. With EA each unit above 256gb is billed as an additional license. See [0] [1] [2].
Re:MongoDB’s New Query Engine
by cj on Thursday September 21, 2023 @05:23PM
(#37604578)
A nice middle ground is Cloud Manager.
MongoDB doesn't like to push the product because profit margins are way lower than Atlas.
Cloud Manager is basically an agent you install on a MongoDB server, which provides a cloud-UI for 1) database configuration, 2) indexes, 3) monitoring/stats, 4) fully managed backups + restoration, and 5) simple GUI for reading/updating objects. Basically everything Atlas does, but the servers are in your AWS account.
I think when we evaluated migrating to Atlas, the cost would have been 3-4x more compared to the equivalent memory/CPU self-hosting on AWS with Cloud Manager.
Re:MongoDB’s New Query Engine
by avereveard on Thursday September 21, 2023 @05:50PM
(#37604898)
Yeah atlas with federated queries and what not makes thinking about the entirety of an application storage layer a breeze. And gpt is even better at generating mongo queries than it is at sql, which is a nice unexpected facilitation in day to day usage.
Re:MongoDB’s New Query Engine
by wackget on Thursday September 21, 2023 @07:16PM
(#37605751)
No support for collation is an enormous dealbreaker. It means you can't have case-insensitive searches or keys (e.g. [email protected] is treated differently than [email protected]). You also can't rename databases.
The query syntax is also a massive pain IMHO. Especially when you get into nesting expressions. E.g. I can never remember if it's `{$id: $regex{'/pattern/'}}` or `{$id: ${regex: '/pattern/'}}` or `{$id: $regex{ pattern: '/pattern/' }}` or something totally different.
Re:MongoDB’s New Query Engine
by willio58 on Thursday September 21, 2023 @07:25PM
(#37605838)
>It means you can't have case-insensitive searches or keys (e.g. [email protected] is treated differently than [email protected]). You also can't rename databases.
Can't you just query for a lowercase version of the input and sanitize data going into the DB so it's only lowercase? I'm not a mongo user but that doesn't seem like a dealbreaker to me
Re:MongoDB’s New Query Engine
by tstrimple on Friday September 22, 2023 @03:51AM
(#37609160)
Yes, you'd do something just like that. Document databases aren't relational databases. People need to think about solving the same problems a little differently. A lot of folks used to relational databases push for some perfect third normal form like disk space is still the major constraint in massive databases. For document databases, you design them based on how they are going to be used. Even if that means duplication of data like storing the lowercase version of something for indexing or rolling up your data into summary collections instead of performing those sorts of queries or collection aggregations on-demand.
Re:MongoDB’s New Query Engine
by hot_gril on Friday September 22, 2023 @03:44PM
(#37616780)
Relational DBs often involve duplication for these kinds of use cases. Simple case-insensitive match can be handled with a custom index that's automatically derived from the col data, but there are more advanced cases where you need manually denormalized tables. Or sometimes it makes sense for performance reasons, but you only do this after you notice the normalized way being too slow.
Re:MongoDB’s New Query Engine
by tstrimple on Friday September 22, 2023 @09:49PM
(#37619936)
Sure. It's not like all folks who lean on relational databases are third normal form zealots. But third normal form zealots exclusively come from a relational database background. Which makes sense.
Re:MongoDB’s New Query Engine
by oron on Thursday September 21, 2023 @12:38PM
(#37600210)
I love Postgres and use it for different projects but for inboxes.com which has a very high insert rate coupled with auto delete by time stamp Mongo up until now has been very kind to me. We sometimes have 1000 incoming emails per second and high usage of our API and it just works.
a) has a proven, supported, easy-to-use horizontal scaling solution. PostgreSQL doesn't.
b) is ridiculously faster than PostgreSQL at per-tuple document updates.
c) has clients which are tailored for operations and data structures around documents.
d) is easier to install, configure and manage.
Those were the marketing points people mentioned in their "honeymoon phase" posts. Then after using it in production, actual benchmarks and comparisons coming in, came the regret and moving on posts. In fact, most of those mentioned moving from Mongo to Postgres, and there was a full blow "yeah, NoSQL was a dumb idea for 99% of use cases, and Mongo even more so" discussion.
In the end MongoDB was the butt of a joke, there were whole memes about it.
So this comment is like a trip down memory lane, or into an alternate universe, where it's like 2012 and these things never happened.
If you are dealing exclusively with documents I find MongoDB to be faster, better and easier to use. If my data model is hybrid or purely relational then I would use PostgreSQL.
After all these years I am still waiting for an horizontal scalability solution for PostgreSQL that is up to the level of a modern database. It's 2023 and for many of us in enterprise environments it's a mandatory NFR.
But it seems like you're not interested in a having an actual technical discussion.
You mean like the "technical" bullet points that could just as well be marketing copy?
In my opinion Citus is still only a hack which is only useful if for some reason you can't move away from Postgres but you have reached the limit of what you can get out of a single machine.
I can implement that in MongoDB such that a query can be done in O(1) whereas implementing this in a relational database requires n joins which in turn requires n foreign key indexes and n row fetches. In many cases this approaches O(n). I have personally seen improvements of 10-100x for this use case when n is large.
Also there is a reason that the standard now in enterprise companies are data lakes and not EDWs. Because the cost and effort to model all of your data is prohibitive especially when 99% of the data ends up never being queried.
Point still remains that PostgreSQL lacks a modern, built-in horizontal scalability solution.
Microsoft on the other hand is among the best in keeping such promises.
If Google had bought it, sure...
Postgres does have things like Citus, or even wire-protocol compatible things like CockroachDB that do scale, but those are not Postgres.
It's the same situation with MySQL vs. Vitess.
sure, citus is an extension of postgres and part of ecosystem.
The writes aren't. There's no sharding for example. You can ever only have 1 primary writer.
I expect the replication story with Postgres will get much better over the next few years.
At what cost though? Does 1 Pg server = 1 Mongo server for the same specs or do you need a lot more Mongo servers?
/s
It was terrible.
It was not fast, a regular SQL db would have been much better, due to bugs in our code, the data in mongodb was not uniform, which led to failures in the code when the document didn't have the expected format.
Cherry on top is that they were mostly growing documents, which is a very slow operation, that would have been done very fast with a many-to-many table in SQL instead.
It supports schemas though right? https://www.mongodb.com/docs/manual/core/schema-validation/
The horizontal replication story in Mongo is just a different game. And because it's so reliable, that ends up being how you solve not just availability and scalability but also engine upgrades and config changes.
I migrated a bunch of production transactional databases on Mongo Atlas from a GCP cluster to an AWS cluster, with zero read/write downtime, by just walking through simple cluster reconfiguration steps. Add some nodes in AWS... move the write master... remove some nodes from GCP. It was stress free and easy.
The same trick with replication on Postgres was a nail biter every time we did it.
does mongodb support ACID transactions? I think this is one of the key question in making decision.
MongoDB’s claim of “full ACID transactions” which “maintain the same data integrity guarantees you are used to in traditional databases” could be misleading. We recommend that users who are accustomed to serializable behavior evaluate critical transactions carefully, to identify whether running at snapshot isolation could violate application-level constraints. MongoDB may wish to revise their marketing language to use “snapshot isolated” instead of “ACID”. https://jepsen.io/analyses/mongodb-4.2.6
;->
(personally i like the schema-lessness)
https://github.com/mongodb/mongo
Edit: ah it's source-available, not open source.
I mean, at least you get access to MongoDB source code, so that is something.
I remember working on a project that used Clusterpoint years ago and the problem was that it was basically an abandoned piece of software due to the company behind it no longer releasing new versions. It was running, yes, but getting data out of it and migrating to something else sure took work and was very annoying because a lot of functionality had been built around it.
So in regards to something so critical as databases and data storage solutions, I'd err on the side of picking things that are open source, or have compatible alternatives with permissive licenses.
If someone were to build their own business around MongoDB (say, offering it as a service) and then SSPL came along, they'd be done for, even if just because of getting caught up in things going on between the org and large cloud vendors or something. Same with how you have to be careful with AGPL, so something like MinIO might be a non-starter in some cases.
Mongo is a thriving database so I’m not worried about it being abandon.
With this, I can totally agree! And as long as it's not AGPL, you should be fine using it for most regular development stuff, even for closed source projects.
that's until provider starts abusing power: increasing rates, close source completely, etc.
I’m really more interested in getting shit done today rather than worrying about 100000 possible future cases. This is how you succeed.
https://www.ferretdb.io/
https://news.ycombinator.com/item?id=23271085
For the idiots downvoting.
2018 was version 11. Right when PostgreSQL JSONB support was beginning to get good.
This authors post has nothing to do with pro/cons of mongodb, and is entirely around a new query engine and computer science. And instead we've devolved into a discussion on the pros and cons of PostgreSQL vs MongoDB.
I get it. People don't like certain tools. But it would be nice for the rest of us to be able to actually discuss the topic at hand without all of the tangential critical noise.
The sub-questions about FOSS and font choice are annoying, in fact I removed my replies on them because I didn't want to feed the problem.
Other than that, I have never seen a proper justification to use schema less db, good for prototyping and get started, not incredible as a long-term solution.
You say "prototyping", but I would generalize that to "faster evolving" in the long term. Of course, not without tradeoffs.
I personally have zero issues writing a custom marshaller when needed for any the schema-less document collections I have. Constraints your application has on the data are a superset of db schema anyways.
[1] https://www.mongodb.com/docs/manual/changeStreams/
[2] https://blog.sequin.io/all-the-ways-to-capture-changes-in-po...
Client libraries are made to do this failover and retry under the hood. Queries are more often made as upserts instead of assuming transactions, eg ObjectId rather than auto increment and so on. Even non upserting operations like $inc are safely retried if handled by queries and not application code.
It's magic to see it happen, with no extra code, the application just keeps running even when one server is down. Starting it up again and it reconciles with no hands-on.
I would love to see a postgres that does this, because relational data is often more useful than the document model.
Please, for sake of the people you want to hear your words. DO NOT USE A SANS SERIF MONOSPACED FONT FOR ANYTHING BUT CODE.
I get it. We stare at text (code) formatted in this kind of font all day long, and many of us find fonts that we truly enjoy. But, most of our monospace sans-serif fonts are designed to make sure individual characters aren't misread. We don't read prose the same way we read code. There is far more pattern recognition going on than actual parsing of individual letters, and monospaced fonts break that. We can debate the aesthetics of serifs but they actually do help provide context clues to the pattern recognition systems in our brain.
Convincing you all to start using serif fonts on prose is not a battle i'm likely to win, but maybe I can convince you to only use monospaced fonts for your terminal, and your code.
Please.
All modern browsers have this.
Contemporary fonts have much less variance in the width of characters, except for a couple of outliers like the i. From there to a completely monospaced font is not as big a leap as you make it seem. For me, I'm fine with monospaced fonts for prose.
Also, the reduced character width (as a monospaced font inevitable has to be spaced at what the widest chatacters, such as w, require, means that you have to use a smaller font size to get the same amount of info per scroll/line/page whatever, again compromising readability.
Here's a comparison between the posted site, and the same site with the font size bumped up 4px, proportional (just browser default, I didn't cherry pick) and the ridiculously wide line spacing reduced.
Even with all that, the easier to read version is quite a bit more compact. Could likely bump the font size 2 more px and still be smaller.
https://i.imgur.com/DBFI2RU.png
I don't want say this in a way that comes off as insulting, but I'll just say it and please don't take it as a put down: if my kid came to me and expressed this much frustration and difficulty because of a sans sarif or monospaced font I would be concerned they had a problem with their vision or an issue processing what they saw, and I might research and/or take them to get checked out.
You probably just don't have a strong sense of aesthetics. Some people notice design-y stuff and some people don't. A lot of people do notice fonts, or misaligned margins, or whatever. And a lot of people don't. The latter category are probably never going to be good designers or artists or whatever. People are different.
What makes you think these proclivities about font choice are universal then? Maybe you are just insisting on what you find aesthetically appealing, while many people would equally validly feel the opposite?
There's an age-old debate of how much "good taste" is subjective, but in this particular case, it's someone hammering a nail in with a wrench. Some will be bothered, others not, but it's definitely the wrong tool for the job.
The example of not liking lights in certain places isn't a very good one. It's an attempt to elide all "things people don't like" into subjectivity, but there are actually things that are better and worse. You would agree that website design can be better or worse, right? And that some people would have a better eye for good or bad design? With me so far? I'm saying you're probably not one of the people out on the end of the spectrum that's best able to recognize good and bad in design, but I hope you recognize that good and bad design exist, and that people vary in their ability to recognize it.
Not to make it personal, but if you are going to claim some kind of innate design ability, you should remove the link to your website from your profile.
Since 5.0 the default write concern is majority, if that's what you were referring to.
https://jepsen.io/analyses/mongodb-4.2.6
[0] https://www.mongodb.com/docs/manual/reference/write-concern/...
From [0] it looks like it was available from 5.1, also interesting is that you can't choose which engine to use, so I suppose it only works on subsets or queries that meet certain conditions.
This isn't the first time I hear something to the effect of "LLVM JIT is great, but it introduces a lot of query latency". I wonder if there are other JIT engines more suitable for compiling potentially small/simple queries.
[0] https://www.mongodb.com/docs/v7.0/reference/sbe/#std-label-s...
https://github.com/bytecodealliance/wasmtime/blob/main/crane...
https://blog.benj.me/2021/02/17/cranelift-codegen-primer
https://jason-williams.co.uk/posts/a-possible-new-backend-fo...
https://news.ycombinator.com/item?id=25130528 30% faster rust build time
There are other specialized JIT compilers that are essentially purpose-built to provide very low latency; for systems like MongoDB which are rarely used for serious analytical processing, you'd probably want to use one of these instead.
[1] https://www.mongodb.com/docs/manual/reference/operator/aggre...
[2] https://www.mongodb.com/docs/manual/reference/method/ObjectI...
There's no limit on number of sortable documents. Use an index to sort and it'll also be fast and use no extra memory. If you want to do an in memory sort there are memory limits, but you can also tell it to overflow to disk.
I've removed that part of my comment. Appreciate the correction.
MongoDB Atlas is a surprisingly good managed database product. I'm not a huge fan of someone else running my databases, but I think it might be the best one you can run across any cloud. If you like MongoDB (and, ignore the memes, there is a lot to like nowadays), and are OK paying a bit more to have someone run your database, I'd strongly consider Atlas.
[0] https://www.mongodb.com/community/forums/t/for-mongodb-enter...
[1] https://www.mongodb.com/community/forums/t/for-mongodb-enter...
[2] https://www.linkedin.com/pulse/mongodb-sizing-guide-sepp-ren...
MongoDB doesn't like to push the product because profit margins are way lower than Atlas.
Cloud Manager is basically an agent you install on a MongoDB server, which provides a cloud-UI for 1) database configuration, 2) indexes, 3) monitoring/stats, 4) fully managed backups + restoration, and 5) simple GUI for reading/updating objects. Basically everything Atlas does, but the servers are in your AWS account.
I think when we evaluated migrating to Atlas, the cost would have been 3-4x more compared to the equivalent memory/CPU self-hosting on AWS with Cloud Manager.
The query syntax is also a massive pain IMHO. Especially when you get into nesting expressions. E.g. I can never remember if it's `{$id: $regex{'/pattern/'}}` or `{$id: ${regex: '/pattern/'}}` or `{$id: $regex{ pattern: '/pattern/' }}` or something totally different.
SQL is still superior.
Can't you just query for a lowercase version of the input and sanitize data going into the DB so it's only lowercase? I'm not a mongo user but that doesn't seem like a dealbreaker to me