I completely agree with you that NoSQL is not a pure-white solution but I disagree that it's only useful for specific use-cases. NoSQL has frequently been touted as this magic concept that makes databases scale well. It doesn't as many of the poorly scaling NoSQL databases on the market today show you. The most you can say is that it maybe lets you ignore a few particularly hard to scale operations. However there's also no reason that a NoSQL database can't scale well it's just that you have to actually design it in such a way that it does and Mongo failed at this.
Joins are a very useful feature of structured databases but there's no reason you can't have them in a NoSQL database. I speak from experience here, RethinkDB has joins (distributed joins no less). They're just as useful in a NoSQL world as they are in a SQL world.
I think the author's analogy "NoSQL : SQL :: Dynamic Typing : Static Typing" (to use SAT syntax) is a good one. I know some damn good programmers who prefer dynamically typed languages like LISP and I know some damn good ones who prefer statically typed languages like Haskell and I think programmers can be similarly productive with either a NoSQL or a SQL databases backing them up. Currently NoSQL has the very real downside that none of the NoSQL databases on the market have really nailed it (ours included but we're working on it) whereas there are some very well established and well tested SQL databases on the market so it's a perfectly rational choice to say you prefer the better tested SQL route. However I contend there's nothing inherently flawed about the NoSQL concept.
By a 'SQL/NoSQL' view, Cassandra has CQL which isn't ANSI standard but close enough to be a SQL.
Google has scalable SQL database called Spanner[1]. They also have a proper NoSQL database called internally Megastore[2] that provides distributed transactions based on entity‐groups on top of BigTable. Also known as High Replication Datastore and available via AppEngine[3].
[1] https://en.wikipedia.org/wiki/Spanner_%28database%29
[2] https://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf
[3] And hopefully via Google Compute pending announcement in few days at I/O. That is if Google actually wants their Cloud to succeed.
I would start with MongoDB, get a grip on what on earth the product is doing, finalize the schema on the fly based on A/B tests, customer feedback and analytics, only once the schema is finalized move it to a SQL database if needed.
If you are not using a good ORM + DB migration system, then MongoDB will make perfect sense when you are quickly iterating through ideas and trying to find a product / market match. You really have no clue how your data schema is going to look like in the end, why confine it at the start? the vast majority of startups sadly won't get even near the phase where they really reach scale related performance issues, so choosing MongoDB for prototyping your business absolutely makes sense to me.
I am curious if this is other folks' experience as well, or do you actually throw out the MVP and start all over at some point? If so, at what point do you make the break?
Premature scalability is as dumb as premature optimization and premature generality.
Hell, in clojure I usually develop against a hash-map of hash-maps stored in-memory until the project gets far enough to re-implement the data layer (which I've hopefully abstracted well enough to not be a big deal).
Think of Mongo less like a schemaless postgres, and more like a persistent redis. Ease-of-use is its true advantage, and I think anyone who exceeds its capacity should be happy to have the problem.
Plenty of personal projects end up running on Mongo, because it has a rich query layer for quick hacking. I also never expect these to exceed the amount of RAM on my machine either.
I've worked for 15 years with relational databases and for the recent 6 months with mongodb on my startup. For me, the agility of not having to define my data in advance, and "work" so much to explain the DBMS how my schema should behave is no less than a game changer in my ability to build the MVP quickly. I just can't imagine having to invest all the labor in having to model everything I did with a normalized database and ORM.
And nowadays it is so easy in a datamodel in a RDBMS to make changes. It is high flexible too.
But another worst thing in Mongo is the performance at complex queries. It is a nice bin for thinks nobody needs or to put documents in but that's it with Mongo.
Mongo is a little different bc they actively position themselves as a MySQL replacement for your average Rails app.
I never liked that rationale:
* The job almost certainly will change. Data lives a long time, usually much longer than the original application.
* It's likely to conflate marketing claims with actual fitness for a purpose. A special-purpose system with special-purpose marketing may sound great if the special case lines up with what you're doing. But that doesn't mean that it's actually a better fit for that special purpose. And, thanks to confirmation bias, people almost always think their situation does sound like the special purpose for which the system was built.
* A database system is a system, not a tool. A tool is used in a particular instance for a short period of time, the results are obvious, and it's easy to switch tools if one is not working out.
Of course, there are always valid reasons to choose the special-purpose system, and I'm not saying you shouldn't have used Couchbase. I just don't like the "right tool for the job" argument.
Another thing that you win with the document model is lots of primary key lookups, making it easy to model your data access around your hot path. http://blog.couchbase.com/performance-oriented-architecture
I just don't like the "right tool for the job" argument.
There is no ONE database that works perfectly in every situation.
Build for what you need now, and just make sure your "central repository of truth" isn't lossy. You can always replicate for specific use cases later.
Even in the short term, it might be much faster to develop against a good RDBMS than a nosql system. I think many developers are so focused on "abstracting away the data store" that they forget what a good RDBMS can do for them.
Very few people need scalability beyond what a regular RDBMS can do. So yes, RDBMS have limits, but you'll most likely never reach them.
All the high-scale users of nosql systems end up having to learn a lot of arcane technical wizardry to make it scale (as demonstrated by the slidedeck linked here), and I'm not convinced it's actually easier to scale something like mongo than a traditional RDBMS.
MongoDB problem is that it promises too much and fail to deliver on a lot of levels. You cant expect to do all the operations one can do on a RDBMS on a NoSQL database and still scale horizontally,while maintaining data integrity. A good NoSQL db should have little features , but features that work very well. I like CouchDB for instance i use it in my apps to store customer infos that need to be synced accross multiple device, but i would NOT build an ecommerce app with it only...
NoSQL dbs solves specific problems. It does not solve the problems relational databases are for.
So selling MongoDB as a replacement for PostGreSQL or Mysql is a blatant lie.It will not work.
It is a fundamentally broken product.
For my own curiosity, what issues caused you the most trouble? The lack of transactions combined with the extremely coarse grained write locking were what put me off the most.
However, one of the biggest issue is the entire sharding design that is incredibly delicate and wouldn't pass even the most basic high availability requirements. I could go into a lot of detail about why it's bad but it would take too long.
After that, it's the hardcoded limitations that prevent true multi-datacentre sharding, the previously mentioned CPU issues with mongos, broken replica selection, inability to control primary/replica setting manually, etc.
The list is virtually endless.
And then I see an presentation like this and think, "Yes, of course, you should have put that stuff in a relational database, what were you thinking".
Of course, most of the open source databases do sharding and replication and the like equally terribly, so it's not a perfect solution....
I'd like LevelDB on MySQL personally.
That statement doesn't make sense to me. One of the first things you do with a sql database when you reach moderate scale is denormalize data so you don't have to do joins, or shard which prevents a lot of joins. Joins are super convenient, but they don't scale.
On nearly all of the cases, they are very usefull, but once in a while one of them gets too slow, so you make a bit more of work enforcing the consistence by hand, accepts a bit more of risk, and denormalize it.
Throwing all of them away because a few may create problems later is a bit of an over reaction.
Here, the naive developer uses MongoDB as an all-in-one that rolls in the part of ORM that too often gets sole consideration, 'How do I persist my objects?', and effectively ignoring everything else. Try as I might to eloquently phrase my frustration with this all, it usually ends up in a rant against those relying on automagic who don't feel the need to understand every decision being made for you.
http://www.h2database.com/html/main.html
It bets MySQL and Postgresql in performance.
I do not hesitate to admit that I was a hipster sometime back too. I chose MongoDB for many of my projects and it went well, till it reached some kind of moderate scale where I realized it was a terrible choice going with a NoSQL db (Sometimes, I'd have to duplicate data because there were no Joins, etc). And that's when you start to realize, that NoSQL is not a pure-white solution. It is designed to satisfy very specific use-cases. Relational databases are really good enough for 99% of the use cases out there.
Unless otherwise you are COMPLETELY unable to design your schema in a relational database, you SHOULD NOT simply opt for a NoSQL database. The claimed NoSQL performance benefits will easily be outrun by a terribly designed schema, if you use the wrong db for the wrong scenario. Trust me, MySQL has had so much negativity because of these hipsters, but even something as basically relational as MySQL scales really reaaallllly well. Infact, many top guys still use MySQL till date, for a reason, in production.[1]
Next time you launch your start-up, spend some time carefully evaluating your db design decisions, as the wrong db for the wrong use-case could easily become the most expensive mistake of your startup.
[1]http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use...