News for hackers. Stuff that matters.
Re:Scaling MongoDB at Mailbox by MichaelGG on Thursday September 12, 2013 @03:06PM (#6376218)
>one performance issue that impacted us was MongoDB’s database-level write lock.

People give MSSQL shit for having row-level locks (if you don't use their MVCC option), yet how is it that Mongo runs with a database-wide option and people don't immediately laugh and walk away? Is the hype so powerful that people just shrug about a huge mutex?

Re:Scaling MongoDB at Mailbox by lucisferre on Thursday September 12, 2013 @03:11PM (#6376249)
The problems people encounter with SQL performance generally (or at least also) are on the read side where query optimization can become a pretty hard problem. Basically, the short answer is people are using SQL and noSQL very differently (as one would expect).

cue: someone telling me SQL query optimization is easy and people are just idiots.

Re:Scaling MongoDB at Mailbox by justinsb on Thursday September 12, 2013 @03:35PM (#6376394)
It's a valid point taken by itself. But I would respectfully suggest that it's not a particularly useful statement, because you're not comparing SQL to anything. "Complex SQL queries are hard" because "complex queries are hard". If there was a magical tool where you could do complex querying and always have the right answer in the minimal time, then these discussions would never arise. (To a debatable extent, that is one of the things people are paying for when they choose a "better" database: the hope that they get closer to the ideal on complex queries)

Personally, I would much rather perform a complex query in SQL than NoSQL. The base-line is likely to be not-terrible, which may be good enough for single-use queries. Optimization is much easier (IMHO) - SQL is a DSL for querying and indexing. Compared to having to write code to do that (which seems to be the primary NoSQL approach), the DSL approach seems much more efficient. SQL isn't perfect, but it beats no-DSL-support or another DSL that is even worse than SQL :-)

Re:Scaling MongoDB at Mailbox by annnnd on Friday September 13, 2013 @05:31AM (#6379398)
The caveat being that in document-oriented store (like MongoDB) the queries tend to be simple(r) - after all, you have no joins, so how complex can they be? Instead of complex queries you need to use appropriate document schemes and/or application logic.

MongoDB has its problems (for instance document schemes can't be enforced, disk space consumption concept is unintuitive,...) and relational DBs have their strengths (ACID, SQL is standard among DBs,...). But very few people ever reach a point where the global write lock becomes a problem. 10gen knows exactly which features are vital to most of their users and which are not...

Re:Scaling MongoDB at Mailbox by justinsb on Friday September 13, 2013 @10:55AM (#6380841)
But you still need to join in NoSQL, for any query that isn't along the natural hierarchy of the data.

The relational model takes this observation and concludes that there should be no "natural hierarchy" in the logical model (the physical model is a separate question.) It's a _theoretically_ beautiful idea. The counter-intuitive _real-world_ result is that the theoretical approach also yielded faster systems than alternative philosophies. I think that's the reason why the relational model has dominated for the past 30+ years.

You might well argue that that's because a _good_ NoSQL implementation has yet to be created (the no true Scotsman argument). That may well prove to be true one day, but I would bet that a good non-relational implementation will be a derivative from the relational world (Postgres with hstore, or Google's F1 with Protobufs); not from a project whose starting axiom is to get rid of SQL/relational.

Re:Scaling MongoDB at Mailbox by lucian1900 on Thursday September 12, 2013 @05:16PM (#6377007)
Yes, MongoDB really is that bad. Never use it.
Re:Scaling MongoDB at Mailbox by tkellogg on Thursday September 12, 2013 @07:40PM (#6377701)
Use rethinkdb instead. Its not perfect but it mainly hits the same niche but does it right.
Re:Scaling MongoDB at Mailbox by lucian1900 on Friday September 13, 2013 @11:52AM (#6381253)
Or anything else, for that matter. Postgres, MySQL, even SQLite beats Mongo for any use-case. Cassandra, HBase if you need a distributed DB. RethinkDB when it gets mature.

MongoDB is in its own class of terribleness.

Re:Scaling MongoDB at Mailbox by akbar501 on Friday September 13, 2013 @01:56AM (#6378898)
RethinkDB looks promising, but it's a bit early.
Re:Scaling MongoDB at Mailbox by dkhenry on Thursday September 12, 2013 @03:11PM (#6376250)
MySQL has a few engines that still have a database level lock ( MyISAM, MEMORY ) and those generally preform better for unbalanced loads ( i.e very read heavy or very write heavy )
Re:Scaling MongoDB at Mailbox by jeltz on Thursday September 12, 2013 @06:00PM (#6377271)
Do you have any source for this? While logic suggests that they should (but perhaps only very slightly) I think InnoDD has got a lot of more optimization work than any other engine.

A benchmark from Oracle shows InnoDB as being much faster on a read-only workload.

http://www.oracle.com/partners/en/knowledge-zone/mysql-5-5-i...

EDIT: By the way the locks are table level in MyISAM and Memory.

Re:Scaling MongoDB at Mailbox by kareemk on Thursday September 12, 2013 @03:53PM (#6376510)
We moved to TokuMX (http://www.tokutek.com/products/tokumx-for-mongodb/) recently and saw a 2x improvement in response times. They've swapped out mongo's backend with their fractal tree storage engine which is MVCC. Definitely worth checking out.
Re:Scaling MongoDB at Mailbox by spamizbad on Thursday September 12, 2013 @04:20PM (#6376673)
This is something I've considered pushing for at my company. Any pitfalls you encountered? Did it reduce db size on disk as much as their marketing claims?
Re:Scaling MongoDB at Mailbox by nasalgoat on Thursday September 12, 2013 @04:36PM (#6376784)
My tests with it showed a drop in disk size of about 20-30%.

Unfortunately the read/write improvements weren't high enough to justify the massive engineering feat it would take to replace MongoDB with TokuMX, mostly due to the lack of commercial support for Toku.

Re:Scaling MongoDB at Mailbox by kareemk on Thursday September 12, 2013 @04:57PM (#6376901)
Our disk size shrank to 10% of stock mongodb size.

The migration is easy now that they've released a tool to replicate from a stock mongo to TokuMX. They also do have commercial support that we've paid for and their team has been incredibly helpful and responsive.

Re:Scaling MongoDB at Mailbox by nasalgoat on Thursday September 12, 2013 @11:48PM (#6378469)
Perhaps once my commercial agreement with 10gen expires in a couple of months I'll take another look.
Re:Scaling MongoDB at Mailbox by yanowitz on Thursday September 12, 2013 @05:49PM (#6377195)
we migrated recently (last week) using the new mongo2toku bridge. it was relatively painless.
Re:Scaling MongoDB at Mailbox by corresation on Thursday September 12, 2013 @07:50PM (#6377734)
People give MSSQL shit for having row-level locks (if you don't use their MVCC option)

Just to clarify, are you talking about MSSQL's tendency to escalate locks (e.g. from row to page to extent to table)? They do that to make locks more manageable, and while in most cases it improves performance, in some high-concurrency situations it can be a gigantic PITA (especially given that the ways to strong-arm it into not escalating is not the most helpful).

Re:Scaling MongoDB at Mailbox by kb19 on Thursday September 12, 2013 @11:48PM (#6378471)
I think that's how it was in previous versions of Mongo but it was fixed in later versions. I'm probably wrong though.
Re:Scaling MongoDB at Mailbox by lucian1900 on Friday September 13, 2013 @04:43AM (#6379297)
It used to be a mongod-level lock, it's now a logical database-level lock. It used totally laughable, now it's merely amusing.