News for hackers. Stuff that matters.
Re:MongoDB and RocksDB at Parse by lpsz on Wednesday April 22, 2015 @12:48PM (#9421691)
MongoDB seems to have a bad rep on HN for its various shortcomings, and yet here FB/Parse seem to use it. Is the bad rep overrated? (Honest question)
Re:MongoDB and RocksDB at Parse by Jweb_Guru on Wednesday April 22, 2015 @12:57PM (#9421762)
No, the bad reputation is not overrated. This is the essence of why appeal to authority is a fallacy. Facebook is (or was) largely written in PHP; that doesn't mean PHP's bad reputation is overrated. Many banks run on COBOL; that doesn't mean Cobol's bad reputation is overrated.

MongoDB's bad reputation is well-deserved and based on a long history of misleading (or outright false) marketing claims coupled with poor technical properties (from unavoidable data loss to horrible compaction performance to global write locks). At this point, even if all the known issues were fixed (and they aren't) it would still take a long time for their reputation to recover. Given that some of MongoDB's direct competitors do not suffer from these problems and can largely function as drop-in replacements, recommending people use something else is really the only responsible course of action.

Re:MongoDB and RocksDB at Parse by spimmy on Wednesday April 22, 2015 @01:01PM (#9421802)
God yeah, the write locks have been a pain in my ass for so long. Fortunately the storage engine API delegates locking to the engine, so we get document level locking in RocksDB or WT.
Re:MongoDB and RocksDB at Parse by lacker on Wednesday April 22, 2015 @02:18PM (#9422399)
some of MongoDB's direct competitors do not suffer from these problems and can largely function as drop-in replacements

Which competitors are you referring to? Personally I find the power and flexibility of the MongoDB query language to be a big positive that for many cases outweighs performance and reliability issues. It seems like no other competitors are quite there.

Re:MongoDB and RocksDB at Parse by functional_test on Wednesday April 22, 2015 @03:53PM (#9423113)
TokuMX supports most of Mongo's features besides the aggregation framework last I looked. Better locking, transaction support, better indices too -- you should check it out.
Re:MongoDB and RocksDB at Parse by benjarrell on Wednesday April 22, 2015 @07:02PM (#9424130)
I don't know that I would call them competitors, but DB2 and Informix both have JSON support and both implement the MonogoDB protocol.
Re:MongoDB and RocksDB at Parse by ahachete on Wednesday April 22, 2015 @07:24PM (#9424208)
Also ToroDB (https://github.com/torodb/torodb) implements the MongoDB protocol. But it uses PostgreSQL as a backend and it's open source ;P

(ToroDB developer here)

Re:MongoDB and RocksDB at Parse by Jweb_Guru on Wednesday April 22, 2015 @10:52PM (#9424771)
In addition to the others already mentioned, http://hyperdex.org/ implements the MongoDB API (though I cannot vouch for all of its claims, it seems worth including for completeness).
Re:MongoDB and RocksDB at Parse by andrewflnr on Wednesday April 22, 2015 @03:09PM (#9422804)
GP may be thinking of RethinkDB.
Re:MongoDB and RocksDB at Parse by functional_test on Wednesday April 22, 2015 @03:53PM (#9423107)
This is the last one of these posts I'll ever respond to, I promise. And I'll give you the same response I've given every other time before:

You should not base your decision of database (or anything else for that matter) on marketing copy. For something as important as your primary data store, you should at minimum read the full documentation and run some tests with dummy data to see if it will even plausibly work for your use case.

I used MongoDB successfully for years with a large data set (>1TB) and 100% production uptime for more than 3 years. I never lost data. Your claim that you will unavoidably lose data is baseless and without merit. In fact, every issue you listed has been fixed, again, counter to your claim.

Personally, these days I prefer TokuMX if I'm looking for something compatible with MongoDB, but these baseless attacks on MongoDB have to stop.

EDIT: Every time I make a post like this, I get some downvotes without responses. Please tell me why I'm wrong. If it's just that I'm abrasive... Well, you would be too if you were addressing the same thing for the Nth time.

Re:MongoDB and RocksDB at Parse by inglor on Wednesday April 22, 2015 @04:02PM (#9423157)
Not the downvoter - but I can totally understand the downvote.

The fact your anecdotal evidence is that you did not lose any data doesn't mean the internet is not full of people who have lost data with Mongo. I have no idea what your workload is, but my experience with data loss and uptime has not been as great as yours.

I'm not for bashing things either - I think there are cases Mongo might be appropriate, I just don't like countering claims with "it worked for me on this one data set". If it drops writes for one out of 100 people that's still a big reason to avoid it if that's a big concern for you. As for "these issues have been fixed" you're welcome to open the issue tracker - no one at Mongo claims all of these issues have been fixed (then again, PostgreSQL has open issues too) so your claim that "these issues have been fixed" is kind of odd...

Re:MongoDB and RocksDB at Parse by functional_test on Wednesday April 22, 2015 @04:11PM (#9423220)
I only brought that up to counter his claim that data loss is inevitable. Of course my anecdote doesn't mean it's not common =) But anecdotes are all anyone else has, and every time I've read one about someone losing data, they either hadn't read the documentation, or just didn't understand the semantics of what they were doing. Very very rarely, especially these days, has it been an actual DB bug (though I will admit I got Mongo to core one time on 2.4 doing a compaction).

And it's a little disingenuous to point at the issue tracker -- as you say, everyone has open issues. The specific things that are mentioned though have been fixed: writes are checked by default now, the global lock has been broken up into per table locks, etc. There may still be common issues that aren't being addressed, but if there are, I'm not aware of them.

Re:MongoDB and RocksDB at Parse by thezilch on Wednesday April 22, 2015 @04:48PM (#9423440)
Anecdotes are not all one has. We have researchers, and it seems you might have missed yesterday's https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-read... (or previous posts on the matter). This link debunks the value of RTFM for Mongo or misunderstanding the API or the rarity of the bug being the DB.
Re:MongoDB and RocksDB at Parse by LunaSea on Thursday April 23, 2015 @03:20AM (#9425351)
I think that it is disingenuous to say that because a fuzzer found certain obscure scenarios where there are issues that automatically everyone is going to be affected by it and that the database has no merits.

Also, if I have to choose a datastore, although marketing shouldn't be important, funding is and MongoDB has had some huge funding rounds in the past. This gives me a lot more confidence in choosing it.

Re:MongoDB and RocksDB at Parse by functional_test on Wednesday April 22, 2015 @10:05PM (#9424629)
I did miss that actually. Thank you for responding to me with something real. I'll reply again once I've had the opportunity to take a look.
Re:MongoDB and RocksDB at Parse by nemothekid on Wednesday April 22, 2015 @08:05PM (#9424312)
If I say "its inevitable that you will get into an accident while drunk driving" and you say "I've been drunk driving for years with 100% no accidents" I would assume you are being dense.
Re:MongoDB and RocksDB at Parse by functional_test on Wednesday April 22, 2015 @10:04PM (#9424628)
That is not a valid comparison, and as someone who has been affected by drunk drivers, I take great issue with your trivialization of a serious issue.

But I'd expect nothing less from HN.

Re:MongoDB and RocksDB at Parse by nemothekid on Thursday April 23, 2015 @01:05AM (#9425102)
How is it not valid? There are documented tests of exactly how and why the database loses data (just like there are studies showing the effects of alcohol), and you have claimed that "it's fine, because it never happened to me". You said the claim was baseless when it wasn't - there is another very popular HN post recently documenting how someone ran a test, proved the database was losing data, and the issue was closed as wontfix (but later reopened). Is aphyr's entire article baseless (and the one he wrote 2 years ago).

In the face of actual data, and reproducible tests - isn't saying something like "well it didn't happen to me" dense?

The comparison might be insensitive, so excuse me for hurting your feelings, but I don't see how its invalid.

A more apt analogy then would be someone saying "My database runs with 100% uptime for 3 years, so there is no reason for me to keep backups"

Re:MongoDB and RocksDB at Parse by functional_test on Wednesday April 22, 2015 @10:13PM (#9424645)
Ah yes, downvotes outnumbering real responses. Classic HN.
Re:MongoDB and RocksDB at Parse by viraptor on Wednesday April 22, 2015 @07:30PM (#9424220)
I promise I'm not trying to troll. Given how the data loss can occur in MongoDB (partitions, silently lost writes) - how do you know that "I never lost data." How do you verify this?

I'm pretty sure that I could kill 0.01% of writes of any random application (which doesn't require extensive audits, like banking for example) and nobody would notice for a really long time. And if the effect was ever noticed, application code would be the first place to look at for the reason.

Re:MongoDB and RocksDB at Parse by spimmy on Wednesday April 22, 2015 @04:03PM (#9423168)
Amen brother. You should never base your db decisions on either marketing copy _or_ HN know-it-all complainypants. :)

We actually evaluated TokuMX extensively last year, pre pluggable storage engine. We might have pursued it if they had implemented a compatible oplog at the time, but with a migration path that consisted of dumping and importing production data -- and no way to re-elect the old primary if there were any production problems -- that made it simply a non-starter for me.

They did eventually implement a compatible oplog, which was a good product decision, but the entire TokuMX engineering team recently quit Tokutek en masse so it's still not a great option in my book. Too bad.

Re:MongoDB and RocksDB at Parse by Frozenlock on Wednesday April 22, 2015 @04:15PM (#9423244)
"TokuMX engineering team recently quit Tokutek "

I would like to know how it happened. Do you have a link to an article or any other source of info?

Re:MongoDB and RocksDB at Parse by mushi on Wednesday April 22, 2015 @06:50PM (#9424081)
They were just acquired, so presumably they may have been running out of money. http://www.percona.com/news-and-events/pressreleases/percona...
Re:MongoDB and RocksDB at Parse by spimmy on Wednesday April 22, 2015 @04:49PM (#9423456)
There's no article, sorry, I just happen to know them.
Re:MongoDB and RocksDB at Parse by takeda on Wednesday April 22, 2015 @05:29PM (#9423704)
> You should not base your decision of database (or anything else for that matter) on marketing copy. For something as important as your primary data store, you should at minimum read the full documentation and run some tests with dummy data to see if it will even plausibly work for your use case.

The issue is that they even lie in their documentation[1].

Also Mongo not necessarily loses data in a catastrophic way, you might have some old or inconsistent data here and there. If you have an authoritative source of data I would compare the data in Mongo with it. Also [1] shows how you can get data corruption even with highest safety settings due to broken design.

[1] https://aphyr.com/posts/322-call-me-maybe-mongodb-stale-read...

Re:MongoDB and RocksDB at Parse by spimmy on Wednesday April 22, 2015 @01:00PM (#9421782)
Yes and no. (That's my blog post btw.) Mongo is a young database, with some super obvious flaws and growing pains. Some of its bad rep is self-inflicted, when Mongo reps make massively overinflated claims about its reliability, scalability, and performance, and then don't back down when the entire internet laughs at them.

But there are some genuinely amazing things about MongoDB and some places where it really shines. Like flexibility -- we run over half a million different apps and therefore half a million different workloads on Mongo. Schema changes and online indexing are painless. Elections are pretty solid. And the data interface layer really can't be beat.

With the pluggable storage engine API, mongo is really growing up and becoming a real database, much like mysql did many years ago when it graduated from MyISAM to InnoDB. I'm excited about the future.

Re:MongoDB and RocksDB at Parse by sanderjd on Wednesday April 22, 2015 @02:54PM (#9422696)
Could you speak (or give me pointers) to how the data interface layer is better than everything else out there?

(By the way, thanks for the balanced, informative, and generally great comment!)

Re:MongoDB and RocksDB at Parse by htsh on Wednesday April 22, 2015 @01:00PM (#9421787)
Yes and no. Though I don't disagree with any of the criticisms or other comments at this level, for some of us MongoDB has been pretty good in production and we haven't experienced data loss. It all depends on your use case.

My first experience with it was for a social game running off of 3 mongo servers in a replica set & here at a large healthcare company we use it for several internal CRUD applications (again in 3-server replica sets) which we continue to iterate. In these use cases, Mongo's flexibility and ease of making changes trumped it's shortcomings. My understanding is that many social games still use Mongo to store player data as our studio was told to use it by a large publisher.

My take is that it's good for building prototypes and it's pretty flexible to both change & migrate data in and out & Javascript devs pick it up relatively quickly. But I also think one should be well aware of it's shortcomings and be careful not to use it where those things are of importance. More often than anything else, I recommend PostgreSQL when other people ask for a general-use db but I'd likely would pick up Mongo myself simply for speed of development.

Lastly, being part of the "MEAN stack," we can point junior devs & interns to one of the many books that cover the stack & they can learn best practices get up to speed quickly. There's an advantage to having books & a slew of SO answers to refer to in that other devs aren't pulled off of their work to teach. We literally had interns committing code on Day 1 of their jobs last summer as they came in having read up on our stack.

Regarding claims MongoDB has made, which I've only been made aware of the last two days, I don't think there's any defending that & it makes my itch to checkout rethinkdb a bit stronger than it was 2 days ago.

Re:MongoDB and RocksDB at Parse by erichate on Wednesday April 22, 2015 @01:21PM (#9421956)
We have been using MongoDB since 1.6 and has worked well for our applications and have not encountered any major issues that would motivate us back to using MySql as our defacto DB.

Knowing the limitations and behavior of Mongo can go a long way in avoiding some of the issues people have encountered.

Definitely looking forward to testing WT and RocksDB in 3.0, beyond performance improvements will drop our storage costs with compression and for a indie studio every dollar counts!

Re:MongoDB and RocksDB at Parse by _cpancake on Wednesday April 22, 2015 @03:47PM (#9423069)
MongoDB is a fun toy database right now, nothing more. Unless you have the resources to fix every problem that comes up, you should use a mature database like Postgres or MySQL (MySQL is not as terrible as some might think).
Re:MongoDB and RocksDB at Parse by threeseed on Wednesday April 22, 2015 @06:18PM (#9423945)
"Mature" doesn't make it necessarily better. I have had catastrophic data losses with Oracle and Teradata and none with Cassandra or MongoDB.

And I would disagree that Mongo is a "toy". We use it as part of our core EDW and we have petabytes of data in our data lake. No issues what so ever.

Re:MongoDB and RocksDB at Parse by takeda on Thursday April 23, 2015 @12:51AM (#9425068)
Can't coment on your experience with Oracle and Teradata, but the danger with MongoDB is that it has non-catastrophic data corruption.

One that you don't even realize until you compare the data with canonical source.

Re:MongoDB and RocksDB at Parse by jasondc on Wednesday April 22, 2015 @12:52PM (#9421731)
For looking outside of HN, this may help: http://db-engines.com/en/ranking