That operating system is Linux. It started out very simple and was good enough for many people and then kept evolving. Nowadays the alternatives are mainly footnotes in history.

MongoDB is also simple. Its locking approach does not give wrong answers. Its users are happy. Your assertion that the locking can't evolve is bollocks because you have no clue how the database works. While your statement has many elements of truth for a relational database, it is meaningless for the current generation of schemaless/NoSQL databases. Did you know that MongoDB has auto-sharding and replication, and doesn't need locks for that either?

How about I call it a tool that holds your data for you and gives it back later. This tool has certain properties that make it work (or not work) for certain applications. It is just that simple.

I do not work with/for 10gen but we have been using MongoDB since version 1.5 and found it incredibly useful addition to our toolbox. It always helps to own more than one hammer.

Fine, I'll bite.

Your assertion that MongoDB was not designed "with concurrency in mind" is simply wrong. They have in fact put a lot of thought into it, they simply made decisions that you either do not understand, or happen to disagree with.

Nobody at 10gen is arguing that a global write lock is the best idea in the world. They are acutely aware of the drawbacks. They initially implemented it that way because it significantly reduces the complexity of offering concurrency in the first place. There's a reason that a NoSQL solution with more granular locking doesn't exist yet. It's much more difficult to construct. There is an ongoing, documented, transparent effort to migrate toward more granular locking, but that is hard and takes time. In the meantime, they have been working on mitigating the drawbacks of their decision by aggressively yielding whenever possible during long running operations.

Did you even read the article? Did you look at the benchmarks? The entire point of it was it's not as bad as you would think, and it's getting better.

And I'm arguing that's going to be a hell of ride, which is more likely to break their backs than yield success.

I read up on the link you posted elsewhere (thanks for that, by the way) and well, it's just as bad as I thought.

Imagine, in pre-2.0 world, there's a greedy reader, and a subsequent writer queued up on the lock. All subsequent readers are blocked until writer quits, which can't quit before greedy reader does. This is a nightmare. That was before 2.0, now the greedy reader will yield, writer will finish, and all the pending readers will be unblocked. This is only an improvement if you consider the nightmarish previous situation. There are still two problems: 1) single writer blocks all readers on a shard while in progress 2) as soon as a new writer is queued up behind the reader lock, all subsequent readers are queued up again.

Does this look like optimal resource use to you? It does not to me.

Let's contrast this with "legacy" engines:

Sybase: readers/writer lock has granularity of a 8kb page. If you're not touching the same page someone else is writing you're fine. (*) they might have moved on since the 1990-s, I haven't looked.

Microsoft: reader/writer lock has granularity of a row. If you're not reading a row someone is writing, you're fine. That was in the 1990s, they have since moved on to snapshots, but have not yet made them default option, I think.

PostgreSQL or Oracle: 1) readers are reading a snapshot and never block writers 2) writers block each other, and granularity of locking is single row. If you're not writing the same row someone else is writing you're fine.

SQL Lite - readers do not block writers, there is a database-wide writer/writer lock. Note that this is a very lightweight desktop-oriented database, not a cloud solution.

MongoDB - reader/writer lock granularity is a shard, the part of the database apportioned to a single CPU core. If you happen to read data on the same shard someone is writing, or is planning to write you're not fine at all. Their plans are "collection level locking".

So I get it, you're saying they planned to add serious concurrency later. I agree on that - they planned. Where you and I disagree is that they will likely fail, because retrofitting concurrency is exceptionally hard. I just can't believe that anyone who knows what he's getting into would actually agree to get into this.

I understand you need to compromise something when you start out, but I think concurrency is the worst possible choice.

In SQLite readers actually do block writers by default. Writing transactions are committed with lock escalation steps. First shared lock is acquired, then reserved, then pending and finally exclusive. Pending blocks new shared locks and waits until all in-flight shared locks are released. Again this is default rollback journal behavior. As of 3.7 write-ahead log allows readers to be concurrent with writers, but AFAIK is still rarely used.

Edit: Turning ON nolock, that is.

I have used Mongo on two projects with reasonably small data sets. My largest collection at the moment is 5 million, and that's basically a log. Other collections are less than 100,000. I've been running mongo 1.6 for a year on these two sites without so much as a hiccup. I do the normal very simple things to protect myself: a cron job to dump db and then copy it to a backup server. And that's it.

I enjoy using mongo for these projects because when I want to add a new feature to one of my domain models, I don't need to think much about retrofitting the data for all instances of that model. I just add an attribute where its needed for the new use case, ensure I have basic checking in my ruby model object and my system keeps incrementally improving.

I think the mongo folks are fantastic in their open dev process and maybe one day, some threshold will be crossed where I can say that for certain types of big data usage mongo is a clear solid choice.

That's exactly the same as adding a new column to your DB with NULL as the default value.

Mongo has the notion of undefined and Null. You can just start putting the new field on new records without having to backfill. Also, you don't have to do the migration thing, which can get messy in big teams (from my experience).

Moving to a doc store from an RDMS really does bring with it an odd sense of freedom when it comes to the schema.

The 'odd sense of freedom' is not always a good thing either. It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea.

Taking the team I worked with at the BBC as an example:

1) There were staging, integration and production environments. Staging and integration would often not be aligned with production, or even one another, because we might find a bit of code turned out not to be production suitable/needed. If this happened we would have to drop the database back to a known, good state. You can't have columns with constraints left around when the code which might have satisfied those constraints is reverted. Doing it without migrations would have been idiotic to say the least.

2) Developers work on different features in different branches, often collaborating. Different features apply new attributes to the db schema. It's important for a developer to know his DB is in the correct state when he starts hacking. You do that with migrations.

Because you almost completely remove the need for schema definition (and what little of it you do, you can do in app. code) you simply don't need the migrations any more. Using mongo means you can pretty much just export your applications domain without having to coerce it into the relational model.

"I don't see what you gain by having both 'undefined' and 'null'."

They mean totally different things. Undefined means that the field has never been explicitly set, null means the field has been set. This means you know what's been backfilled and what hasn't - you can't tell without extra metadata in mysql. Also, in mysql if you provide a null default then every row has to be updated.

"It's like BASIC allowing you to use a new variable without declaring it. It may be convenient but nobody calls it a good idea."

I don't know BASIC but you can put Perl into a certain configuration that allows this. That makes for horrible scoping issues that aren't analogous or applicable to what we're talking about.

In BASIC you can 'declare' a variable by simply using it. The compiler will not warn you if you use an undeclared variable. That's the analogous situation here.

Actually Mongo is bad for really humongous data sets.

It works well if the working data set (the data you commonly need) can fit in memory.

Of course this doesn't scale very well with say several terabytes of data, while there are Oracle databases that handle a lot more...

In the case you Mongo you go to sharding etc and things get complicated in your app handling.

Can you elaborate on this?

It's still quite bad -- the 2.0 benchmark shows a dropoff of 1000 read qps with as few as 60 faults per second. The 1.8 metrics drop to damn near zero reads under the same scenario. That's obviously worse, but a 33% drop in read capacity for 60 faults/sec is bad, no matter how you look at it.

Another way of thinking about it: if you're throwing 3,000 qps at a mongo instance, and expecting only 60 faults/s, then you need to ensure that 2940/3000 (98%) of your working set fits in RAM. Maybe there are toy problems where that's true as the DB grows to terabytes of persistent storage, but there aren't very many.

At one point the mongodb guys were talking about moving to a record backend that looked a lot like any traditional relational database. Why would they do that if they felt that they had made the right architectual choice in the first place? More importantly, does anyone really believe that the tradeoffs in their current design are necessary?

In other words, your analytical faculties have been intentionally shut down.

I am pretty confident that a system not designed with concurrency in mind cannot be retrofitted with any decent concurrency later.

I think you're probably right from a software engineering perspective. But you haven't actually shown that Mongo was "not designed with concurrency in mind".

I've implemented in-memory persistent storage systems with a global RW lock. Depending on the characteristics of the load it can be ridiculously fast. E.g., on the order of tens of microseconds for typical transactions.

Complex schemes like MVCC with multimaster replication are an amazing accomplishment and are great for general purpose SQL-driven databases. But it's not the only reasonable design choice in the world and it's certainly not the optimal one for every problem.

For a sign of things to come, I invite you to take a look at how relational database vendors are fighting to squeeze single-digit percentages from their engines to beat benchmarks such as TPC-C or TPC-E. NoSQL will get there too - fight for efficiency.

Did you really read the post before arguing this?

mongod threads yield their lock (read or write) in two classes of situations:

* yield-on-page-fault – v2.0 implements a yield-on-page-fault feature which results in much more concurrency than one would achieve with a pure reader/writer lock. For common operational cases, file system page faults are detected in advanced and handled outside of any lock, then the lock is resumed. Not all fault situations yield, but many do. This results in v2.0 having much better concurrency in practice than v1.8.

* yield-on-long-operation – mongod also yields periodically on common operations that are extremely long running. The goal here is to allow interleaving so that other operations which are quick-running can execute soon.

http://www.mongodb.org/display/DOCS/How+does+concurrency+wor...

And there's where your legacy understanding fails. You're really not supposed to be doing a lot of disk IO in modern datastores in the first place. Your working set should be in RAM.

> beat benchmarks

If my vendor is investing time in beating benchmarks instead of solving real problems, I'm finding a new vendor.

I am not at all addressing MongoDB here to be clear -- just your comment regarding worthwhile "problems" to "solve".

It is not too difficult to foresee a future where energy costs will trump all other considerations, including development time, sys ops, etc. Specially for <X>aaS efforts, energy efficiency can clearly end up being a competitive edge.

it was designed with concurrency in mind, the concurrency you're looking for comes in the form of sharding and replica sets though.

edit: meh, got bored, decided to defend it: http://news.ycombinator.com/item?id=3412283