News for hackers. Stuff that matters.
Re:MongoDB Wire Protocol Specification License by gentleman11 on Wednesday July 21, 2021 @01:44PM (#27909489)
Out of curiosity, why did Mongo get so popular? I used it in university when it was hyped and it was pleasant to get started with. The ability to query your data is so inconvenient though, I switched to SQL and I'm confused about what made experienced developers switch from SQL to mongo in the past. Is it just because you can scale up more easily when your database gets massive?

Like NodeJS and many web technologies, almost everything I read about it years ago turned out to be hype and not based on facts (eg, nodejs being faster for large numbers of requests than traditional backends is not true, but it was widely repeated on the first 10 pages of google search results)

edit: very well said, thanks for all the replies!

Re:MongoDB Wire Protocol Specification License by MildlySerious on Wednesday July 21, 2021 @02:14PM (#27909914)
Mongo popped up around the time node started growing more in popularity. Around 2012, 2013 I started hearing it everywhere, and it felt like a hard pressed marketing attempt to make Mongo the database for node, and by extension the database for the internet.

The web-scale meme came from them actually trying to position themselves like that, and it partially worked given that it is not only still around, but the prevalence of the MEAN/MERN stack, at least in (tech) pop culture and social media.

To this day my opinion is that Mongo is a snake oil PR firm with a "database" and I have not yet seen anything that has convinced me otherwise.

Re:MongoDB Wire Protocol Specification License by rapsey on Wednesday July 21, 2021 @01:48PM (#27909549)
Because like NodeJS it has a low barrier to entry. This is by far the most important feature for rapid product uptake.
Re:MongoDB Wire Protocol Specification License by mcintyre1994 on Wednesday July 21, 2021 @01:49PM (#27909557)
I think a lot of it is how easy it is to insert, query and update nested objects compared to a normalised SQL database. I know that Postgres' JSON field has come on a lot in the mean time though.
Re:MongoDB Wire Protocol Specification License by mjburgess on Wednesday July 21, 2021 @01:50PM (#27909572)
Rapid application development.

During which, data is naturally hierarchical and schema-less (ie., not defined, changing).

Mongo makes it easy just to "save your dictionary"

It's really just an "sqlite for hashmaps", and should have remained this.

Re:MongoDB Wire Protocol Specification License by fennecfoxen on Wednesday July 21, 2021 @04:07PM (#27911428)
More specifically, it's an mmap()ed series of linked lists of BSON documents, with some relatively simple B-tree indexing, a little journaling, and an unfortunate tendency to drop data on the floor.
Re:MongoDB Wire Protocol Specification License by bogomipz on Wednesday July 21, 2021 @07:45PM (#27913594)
Mongo hasn't used the MMAP storage engine for a long time. In fact it looks like it has even been removed now:

https://docs.mongodb.com/manual/core/storage-engines/

Wired Tiger, the default storage engine is LSM Trees with BTrees for indexes.

Re:MongoDB Wire Protocol Specification License by matchy on Wednesday July 21, 2021 @08:00PM (#27913695)
Not LSM, all BTrees, but the rest is correct.
Re:MongoDB Wire Protocol Specification License by bogomipz on Wednesday July 21, 2021 @08:48PM (#27914058)
Not sure of the current state of Mongo but WT certainly has LSM Trees:

http://www.hpts.ws/papers/2013/WiredTiger-HPTS.pdf

http://source.wiredtiger.com/3.2.1/lsm.html

Re:MongoDB Wire Protocol Specification License by whydid on Wednesday July 21, 2021 @06:41PM (#27913086)
Junior developers are terrified of their application running slow, and not having the technical aptitude required to debug and fix the performance problem.

Additionally, Junior developers do not care about data integrity (the customer's problem), or operational complexity (the SRE's problem).

Finally, Junior developers consider software to be much like bread. Older software is bad (such as SQL relational databases), and newer software is good (NoSQL).

Combine all three of these flawed systems of thinking, and you have MongoDB adaption.

... also before Kafka, the defacto queue broker was ActiveMQ, which was a pain to deploy, so many people used Mongo as a queue.

Re:MongoDB Wire Protocol Specification License by gbuk2013 on Wednesday July 21, 2021 @02:24PM (#27910058)
(Just finished their M121 course for my newish job).

Their aggregation pipelines are pretty pleasant to use. You basically get a bunch of data transformation code that you can avoid writing and you can work with your data without it going out on the wire.

I would probably be able to do most of that in SQL and I’m sure someone who really knows SQL can do everything it can do but I suspect the query involved will be significantly gnarlier. :)

Not sure about how the two would compare performance wise but I do know that you can’t use “$lookup” (Mongo equivalent of joins) with sharding which strikes be as quite unfortunate for scalability.

Re:MongoDB Wire Protocol Specification License by vel0city on Wednesday July 21, 2021 @04:00PM (#27911334)
You don't use a screwdriver to drive a nail, and you don't user a hammer to drive a screw. Its good to use the right tool for the job. In many instances, a relational database (often SQL) makes a lot of sense and is the right tool for the job. Some workflows, a NoSQL/Non-relational database design can be nice. Document databases, K/V stores, etc can be more optimized for those specific use cases which can lead to some better experiences. That said, lots of relational databases are quite powerful and quite performant that unless you really need the particular use cases the NoSQL databases focus on, it may just be better to use the tooling available in your relational database.

There are a few instances where I've really preferred using a document database. Things like logs are often really useful to have in a document database like MongoDB. In one collection I can stick all kinds of logs. The schema of all the different fields in the document are really relevant to what kind of things I'm logging. If I were to have a column for every field I might have in the log message, each row would have a ton of null columns. If I wanted to add a new field to easily search on, I might have to add a new column. A "schemaless" design can then be very useful. Of course this is true of pretty much any document database and not necessarily only MongoDB, and these days some relational databases have decent JSON document database functionality as well. There are always certain kinds of tradeoffs to be made when choosing one database technology over another.

Another thing I've found useful with MongoDB in particular is GridFS, that has been pretty neat to use. These days there are other systems of handling binary blob storage, but as mentioned earlier there are always tradeoffs with choosing any particular technology.

Re:MongoDB Wire Protocol Specification License by tayloramurphy on Wednesday July 21, 2021 @02:17PM (#27909972)
Reading why RethinkDB failed may provide additional context as to how Mongo was successful.

https://www.defmacro.org/2017/01/18/why-rethinkdb-failed.htm...

Re:MongoDB Wire Protocol Specification License by bogomipz on Wednesday July 21, 2021 @08:32PM (#27913934)
I believe it became popular due to the following:

It had a flexible data model as there is no predefined schema. The selling point was rapid protoyping.

The JSON-based documents meant developers could work in objects without an ORM. If I remember Mongo called this "fixing the data impedence mismatch." Sigh.

It had decent performance as long as your working set fit in RAM as it was basically a mmap()'d file.

It was easy to get up and running.

It was in the right place at the right time during the original NOSQL hype cycle.

Re:MongoDB Wire Protocol Specification License by ithrow on Wednesday July 21, 2021 @01:55PM (#27909649)
IMO, nodejs today is a reliable tech for many different kinds of server side workloads, I wouldn't compare it to mongodb.
Re:MongoDB Wire Protocol Specification License by gentleman11 on Wednesday July 21, 2021 @02:14PM (#27909915)
I wouldn't say mongo is unreliable, and I enjoy node a lot. Its just that, years ago, everybody said it was amazing for certain things and it wasn't based on anything
Re:MongoDB Wire Protocol Specification License by flowerlad on Wednesday July 21, 2021 @01:50PM (#27909580)
SQL relies on joins. Joins are fine for reasonably sized databases, but on internet scale joins have miserable performance. "Document" based databases such as Mongo are designed to avoid joins.

MongoDB has automatic sharding, which is very important for horizontal scaling of writes. For MySQL you have to do manual sharding which is extremely hard.

And finally, document oriented databases are schemaless, which means there is no downtime when you add/remove fields.

Re:MongoDB Wire Protocol Specification License by McGlockenshire on Wednesday July 21, 2021 @03:18PM (#27910777)
This entire post sounds like the "MongoDB is web scale" video that is now a decade old. "Internet scale" is meaningless jargon. "Joins are slow" is a meme based on now-ancient MySQL performance.

Schemaless / document databases have a place and a purpose, but nine times out of ten the data we're dealing with day to day has a known and rigid structure that changes infrequently.

Re:MongoDB Wire Protocol Specification License by gbuk2013 on Wednesday July 21, 2021 @02:27PM (#27910102)
That’s the theory, in practice “no schema” in my recent painful experience translates to “move the schema into the apps” which really sucks when you have a bunch of interdependent services owned by different teams.

And you still need to do joins to enrich data from different collections.

I want Postgres back. :(

Re:MongoDB Wire Protocol Specification License by laichzeit0 on Wednesday July 21, 2021 @03:00PM (#27910541)
It’s exactly that. Shifting the problem from schema-on-write to schema-on-read. It’s like dynamic vs statically typed. Personally, I don’t like it at all, none of your problems have magically disappeared, you just shifted it to runtime (or the application layer, as you say).
Re:MongoDB Wire Protocol Specification License by vel0city on Wednesday July 21, 2021 @04:05PM (#27911401)
In many document databases these days you can have required fields and types enforced by the database to ensure all documents in the collection have at least a certain schema.

https://docs.mongodb.com/manual/core/schema-validation/#rest...

Re:MongoDB Wire Protocol Specification License by flowerlad on Wednesday July 21, 2021 @02:37PM (#27910218)
> really sucks when you have a bunch of interdependent services owned by different teams.

That's true. The strictness of SQL lets you use SQL as the integration point. Without it you'd have to write a service on top as the integration layer.

Re:MongoDB Wire Protocol Specification License by jjav on Wednesday July 21, 2021 @06:43PM (#27913097)
> “move the schema into the apps”

Exactly. There is always a schema, there has to be. It's just a question of whether it can easily be seen or if it is buried in a thousand places in the code.

Re:MongoDB Wire Protocol Specification License by fredros on Wednesday July 21, 2021 @02:36PM (#27910214)
> but on internet scale joins have miserable performance.

Please define internet scale. Joining 10s of billions of rows using clickhouse here.

Re:MongoDB Wire Protocol Specification License by flowerlad on Wednesday July 21, 2021 @02:38PM (#27910226)
Clickhouse is OLAP. We are talking OLTP.
Re:MongoDB Wire Protocol Specification License by ksec on Wednesday July 21, 2021 @02:45PM (#27910336)
>For MySQL you have to do manual sharding which is extremely hard.

Vitess [1] and PlanetScale [2] ?

[1] https://vitess.io

[2] https://www.planetscale.com

Re:MongoDB Wire Protocol Specification License by bogomipz on Wednesday July 21, 2021 @08:04PM (#27913720)
>""Document" based databases such as Mongo are designed to avoid joins."

That is not something specific to document databases that is an approach taken by NOSQL datastores. It's known as denormalization which has its own share of issues that then need to be addressed.