After several years of pain e.g. cursor timeouts (even when supposedly disabled), iterating over the same document multiple times in the one query etc. and of course no transactions (since added). I eventually migrated to PSQL (using some custom scripts) and everything is so much easier to maintain, much more stable and much faster.
We're still using MongoDB for our analytics though, as it's inherently unstructured non-relational data, so MongoDB does actually do an okay job. Well, except for the run away memory usage which leads to k8s killing the DB server for violating memory constraints every so often. Yes, MongoDB itself is configured with memory constraints, it seems WiredTiger ignores them.
Licensing was considered, however honestly MongoDB itself is (or at least was) so bad that licensing wasn't the primary motivating factor.
EDIT: Just to clarify, most of our data is relational, however we do have some unstructured data in PSQL stored in JSONB columns. They work a treat and in some cases have complex indexes on them - they far out-perform anything MongoDB was able to offer us.
Why do we do it? Cost saving. It's that simple. For your information, I wouldn't say we do it "properly" either. We do a decent job though.
Basically, for us (and many other businesses) DBs are not the bottle-neck. Our business model facilitates a heap of static content that's served (via a CDN) without even hitting the DB. When the DB is used, it's 99% read-only; except for the analytics which aren't business critical.
As with all things, it's a trade-off.
Maintaining VPS infrastructure just to run a DB is a huge amount of overhead. More so, when the DB crashes and/or wildly consumes memory. You need disk/memory monitoring, logging, process monitoring, availability monitoring etc. What if the VPS itself goes down? What automated system is going to spin it back up?
A self-managed VPS DIY solution is substantially more expensive to maintain than using well-established tooling like k8s. These are all already solved problems. Not just that, it's a standardised work-flow with the rest of our infrastructure i.e. no manual futzing around with VPC etc. to get our cluster and externally managed VPS talking to each other.
For us, a DIY solution was never even on the cards. It was either a fully managed service or what we have now.
However, fully managed services don't just cost more per month. There's frequently additional development overhead, as the offerings are "off the shelf" software and not necessarily easily customised. For example, right now our PSQL deployment is running a custom PSQL extension that seamlessly handles BSON (MongoDB) Object IDs, semantically converting them to UUIDs. Thus, post-migration from MongoDB, application code isn't riddled with code handling legacy IDs.
Monthly service provider fees are but one consideration when performing costings.
EDIT: It's probably also worth noting that our entire infrastructure, including DBs, can be spun up as a k8s cluster on a developer's work machine in one command. We of course also have a staging environment, which mimics production. Development also ought to mimic production as closely as possible (but still be debuggable).
Are you saying companies are hurt by the license, or are you saying companies are hurt by software's behavior (e.g. consistency model)?
Then their sales people started coming around and asking for insane sums of money (like $10,000 per instance per year plus 100% markup on hardware to run in AWS - support is extra). They were worried about amazon and others offering a better so next they changed the license.
If it had been SQL it would have been easy to drop in another database. But Mongodb is so different from anything else that you need a major rewrite to get away from it. Postgres has some similiar functionality but it’s not drop in. Amazon has a more or less public beta of DocumentDB which as far as I can tell is a mongodb compatible interface to aurora backend - it works but doesn’t have the polish of RDS yet.
> If it had been SQL it would have been easy to drop in another database.
There’s no incentive to not extend or try to improve SQL semantics because for proprietary databases it increases lock-in which is good for sales and for OSS databases the extensions are genuinely positive and increase developer productivity and you don’t feel bad about locking your users in because it’s OSS.
We tried to do a migration from Sql Server to Postgres.
Even went as far as building a parser/compiler to transform Sql. Certain things are just too difficult to handle.
1. Sql Server supports arbitrary sql statements in queries. Postgres does not.
2. Data types are not the same.
3. Feature sets at the 'interface edge' between the client library and the engine differ. For example, Sql Server queries can return an arbitrary number of results.
It's true that Mongo uses a custom language, but with Mongo - you are unlikely to have any significant business logic in your data layer.
Also their API surface is relatively simple.
For example, FoundationDB has a layer that mimics the Mongo API. Amazon has one as well.
More likely, though, is that Oracle users use Oracle-specific flavors of SQL in lots of legacy apps that no one wants to touch, and rely on the support contracts for continued assistance in keeping the database performant for its query load.
To really take advantage of a database engine, you will probably end up using engine-specific idioms.
Postgres and Sql Server support recursive CTEs. MySql probably doesn't?
Sql Server supports cursors but is dog slow and generally not recommended for any sort of large scale operations.
Isolation levels and locking are really important as well. Do all engines even support the same isolation levels?
If the companies got lured into using MongoDB as a service, then I understand that prices could increase and companies could be in a pickle. But if companies got lured into using MongoDB as software, I don't see how they could have a problem.
The parent comment states "their sales people started coming around and asking for insane sums of money" but I can't find any reference to this.
If MongoDB really is sending salespeople to make claims against companies who they convinced to use their OSS database as a data-store, than they are indeed abusing their license and no one should use their database or support their company.
From what I read of the license, that shouldn't be the case though.
The only major change to the license is in adapting to the rise of cloud services.
That's one of GPL's biggest strengths.
Also successful software packages that are GPL are GPL for everyone. You don't have a mother company that can bypass the GPL due to them owning the copyright. And when you do it's usually a scam that people can smell.
You can't say about a license change that it is in the "spirit of GPL" when the mother company, MongoDB Inc, doesn't have to play by the same rules as everyone else.
So I'm sorry, but when you're talking about the "spirit of GPL", you don't know what you're talking about.
The FAQ for the license change very clearly indicates that your reading is what was intended. Question #4 particularly contains the complete text of the section that was altered, and it's hard to see how they could have left themselves any room to penalize regular users.
Well actually....
One corner case. If you are using Mongo with the Mongo Linq driver in C#, you can switch it out for an RDMS without changing too much code with just a little foresight.
I think it’s just going to take somebody shaking the cobwebs off of their X.500 manuals and making the query syntax and schemas a little more ergonomic for them to come back en vogue.
It’s one thing to have the ability to mold your DB to model hierarchical data but have your DB engine support it non-awkwardly might be a game changer. People are already used to apps needing a DB, cache, queues, S3, filesystem, service discovery, that adding one more thing would hardly break the bank.
When I did implement a system with C#/Mongo, I fell in love with changing my database schema just by changing my C# objects. With the C# driver you can include a Dictionary as part of your object to round trip unmapped fields.
Not to mention that after you go through all of your modeling and figuring out your aggregate roots, you can just store the entire object graph with one .Add and the entire object is stored atomically without having to worry about transactions, table locks, etc.
Unlike with an RDMS where your beautiful aggregate root has to constantly be composed and decomposed.
Why use a relational store? If you’re working with a system that will only be read and written by an object based language? Why keep converting back and forth between a relational model and an object oriented model?
In C#
var seniorMales = from c in context.Customers
where c.Age > 65 && c.Sex == “M”
You get autocomplete and compile time checks.
When you want to add an record to your Mongo collection, you work with strongly typed
IMongoCollection<Customer>
Well I can't tell you what the right choice in your system is, but in the general case I think going with a relational model by default generally makes sense because it's rather future proof. By that I mean a relational model gives you the most flexibility in the future to evolve your system in ways you couldn't/didn't anticipate when the system was initially designed.
No point using a relational db for that kind of thing.
Not to mention the LINQ driver....
And you can’t do updates on individual JSON fields in Postgres.
The problematic part with PostgresSQL JSONB is modifying a JSONB column content. In any case, we totally replace the old content by a new one. So, the update operations turn to the complex queries that can lose the content. You can avoid this by performing the full JSON reorganization before the persistent operation.
You still don’t have the richness of MongoQuery and the aggregation framework and the tooling around working with JSON data isn’t there. Let alone the native driver support.
Anything that you can store in object database you can normalize and store in an RDMS. ORMs have been translating relational models back and forth to object models for decades.
In the case of Mongo, you just create one object containing other objects, lists, arrays etc and the driver serializes it to a JSON like structure and stores it as is in the collection.
In the case of an RDMS, you create your C# classes to model your tables and relationships.
The driver translates the LINQ to the appropriate query language.
Hahaha. No.
If anything, migrating from something like Mongo might possibly be easier. Because it's non-relational, you probably have very little business logic in your database layer.
Unlike with Sql.
And even though Sql has a standard syntax, beyond vanilla CRUD queries, there are many differences. Even if 2 engines support the same synatx, the idiomatic way of doing something will differ.
(For example, cursors in Sql Server is extremely slow, whereas its a perfectly acceptable way of doing list operations in Postgres)
CouchDB is quietly weeping in a corner.
Additionally I'm curious how the license is harming companies, as I mentioned in this comment: https://news.ycombinator.com/item?id=20862996
MongoDB had a good run, I enjoyed it before migrating away. This license definitely hurts companies.
What to? I built my own Open Source solution (MIT/Zlib/Apache2).
Around same time Firebase was getting popular, and Graph databases were the way forward.
Combined them all together, now have Internet Archive and HackerNoon running it (https://github.com/amark/gun) in production!
Link to tech talk for others: https://www.youtube.com/watch?v=BEqH-oZ4UXI
Could you elaborate on how companies are hurt by the license?
I had the opportunity to redesign the architectures of few productions running apps because of this superstition "Companies are falling due to MongoDB, lets switch to another DB"
Every single time, as an architect I have found that developers are just being fancy about their first-hand experience with some other database and they are unable to think in new direction which MongoDB is built for.
Following are the things most people miss:
1. MongoDB is not safe/secure. Anyone can hack it -> the default configuration were not optimized in previous MongoDB version but it doesn't mean it lacked User security API.
2. MongoDB does not support JOINs or query API just sucks -> No, it doesn't. First, You need to learn about denormalization and schema handling. Second, MongoDB supports JOIN semantics such as $lookup & $graphLookup. Third, it supports ElasticSearch like text searching capabilities and scoring. Fourth, it supports data aggregation pipelines with various ways to compose the data transforming operations including above 2.
3. It does not have triggers like RDBMS -> It has "change streams" which is more flexible than Trigger. This feature also enables multiple application servers in microservice architecture to use MongoDB as a central store. Want a custom ETL streaming pipeline? You got it.
4. It does not scale like ABC or XYZ DB -> Again understand point #2. Also, look into docs how distributed sharding works across replica sets. Spend some quality time figuring out what should be the KeyField to be used for distributed sharding of Data.
5. Licencing sucks -> Have you understood the whole project cost and complexity ? Have you actually looked at all the terms and conditions or are you just a big fan of OSS and for that you can support any stupid alternative?
Also, you got downvoted by the same people that upvoted the stackoverflow post on the problems of downvoting on technical problems on the internet.
People might not like or know the technology and are guided by older tech gurus from the previous generation such as Edgar Frank "Ted" Codd, although he was a genius, now the market is much larger and there's a lot more use cases.
Additionally, most people know RDBs such as mysql and that's what they are comfortable with, i.e. can't get out of their comfort zone and give an honest and good try to something else or don't even have time.
Also it's what's taught in courses as superior to non relational databases.
I would not change our MongoDB clusters for Postgres/MySQL/etc even if I could do it with a single command. In this case it's not relational data though, for a large business relational model I would stick with RDBs.
And I am still yet to find any database that even comes close to its single tuple update speed.
2) Of course. But MongoDB is orders of magnitude faster in some cases, has better horizontal scalability capabilities, better drivers and some nicer semantics e.g. around streaming.
https://hn.algolia.com/?sort=byPopularity&prefix&page=0&date...
No hate, just always amused me that someone on HN had MongoDB as their hobby horse.
I stored about 2KB of data for 72 hours and got charged fifteen dollars before I realised the mistake.
They really need to come up with a per mb per hour option.
I evaluated DocumentDB and price wasn’t the issue (at our scale, the price is OK).
Main blocker is that it’s locked at Mongo 3.4 API compatibility, which is several major versions behind. But worse, their implementation of the 3.4 API is incomplete, so while they advertise “Mongo Compatible”, it really isn’t a drop in replacement.
Our Security, Cloud Engineering and Finance teams understand VPC, IAM, Costs etc back to front and so when you add a new AWS product they know how to manage, secure, finance and support it. And of course there is no Procurement process with adding a new AWS product.
With a managed MongoDB (even though it's on AWS) you need to get buy in from dozens of people and go a vendor comparison to get through Procurement. It's why for many companies AWS dominates and in the future could well end up owning everything under the application layer.
There's a common saying we have in our consulting circles, that "Processes should support business, but business decisions shouldn't be made because of lack of processes".
We had a large local bank choose not to pursue a good opportunity because "our procurement process takes too long". This instead of investing in fixing the procurement process.
What happens if a company that uses Software A has good motivation to use B, but their support functions don't understand B? Aren't the support functions supposed to improve by seeking to understand B, even at the initial inconvenience of time and resources?
In a .gov environment, we literally paid 5x more for certain services because the contract terms demanded were too costly or onerous for OEMs to handle. So everything funneled through middlemen of dubious value, who basically borrowed money, pushed paper and carried insurance for a vig that pushed up the price.
The procurement people were very happy, because they got their three bids that varied less than 1%.
Also, Amazon doesn’t have the same reputation as Google for killing products, so it’s a pretty safe bet. And AWS will be around for quite a long time. The financial stability of MongoDB (the company) isn’t as guaranteed.
Maybe the parent’s company’s processes did work in this case.
It's not about the safety of the bet, because from an OSS perspective, the issue seems to be that many are moving away from MongoDB to something even more opaque. I don't follow AWS, but do they publish a roadmap of planned changes in their document DB?
As much as I love a managed service this was a very disappointing effort from AWS.
But we can do Hunger Games too. There's no wrong answers.
I'm not enough of a DB person to know if it was Mongo, or the ORM I was using (Waterline), but my data is extremely structured and relational. Once I switched over to Knex/Objection with a Postgres DB, everything went much smoother. The transition, though, took around two months, was hell, and I almost turned back many times.
>... they continue to build on top of a broken foundation (MongoDB) ...but last I heard Stripe was still running a very outdated version ...