News for hackers. Stuff that matters.
Re:10gen raises $20m for MongoDB by deweller on Monday September 12, 2011 @10:49AM (#2987473)
From what I've read, I wouldn't pick MongoDB my first choice for a large data warehouse. Maybe 10gen can change that.
Re:10gen raises $20m for MongoDB by colonhyphenp on Monday September 12, 2011 @11:50AM (#2987726)
Can you go into some more detail why you would not choose it? I am curious.
Re:10gen raises $20m for MongoDB by deweller on Monday September 12, 2011 @12:16PM (#2987820)
I don't have facts to back up my opinion - It is formed by memories of articles I've read.

I believe CouchDB is a better choice for very large data sets because of its design.

+ CouchDB uses a Map Reduce design that I believe would scale better over very large data sets. + CouchDB always stores data in a consistent state on disk. You can literally pull the plug on the server at any time and the data will never be inconsistent.

MongoDB is geared for performance and is a great bridge between a relational database and a high-performance No-SQL database. But I don't recall that it's strength is handling large datasets.

Re:10gen raises $20m for MongoDB by ethangunderson on Monday September 12, 2011 @12:25PM (#2987843)
Comparing map/reduce in Mongo and Couch is really apples and oranges. They are designe to do two different things. i.e data processing vs building views.

Mongo is designed from the ground up to deal with large datasets. Take a look at their sharding architecture.

Re:10gen raises $20m for MongoDB by catch23 on Monday September 12, 2011 @01:37PM (#2988134)
I guess it depends what you consider "very large". If you're talking about multi-petabyte, then I'd probably use hdfs, but otherwise mongodb might fit. I hear craigslist uses mongodb to store their data since 1997, which is a fair amount of data I believe.
Re:10gen raises $20m for MongoDB by jzawodn on Monday September 12, 2011 @05:34PM (#2988998)
Uhm. The data goes back that far but the usage is far more recent than that.
Re:10gen raises $20m for MongoDB by Wijnand on Monday September 12, 2011 @03:31PM (#2988576)
1997, are you sure?
Re:10gen raises $20m for MongoDB by simonw on Monday September 12, 2011 @04:51PM (#2988854)
Re:10gen raises $20m for MongoDB by duggan on Monday September 12, 2011 @06:42PM (#2989170)
When he says "store their data since 1997" I believe he means "store data dating from 1997 on" - slight confusion in choice of wording!
Re:10gen raises $20m for MongoDB by alnayyir on Monday September 12, 2011 @01:20PM (#2988064)
CouchDB is better'ish for larger datasets, but not for arbitrary scaling. MapReduce in CouchDB requires dumb full-scans if you're not just refreshing an existing view.

Arbitrarily large data is the exclusive domain of hadoop/hypertable/cassandra AFAIK atm.

Re:10gen raises $20m for MongoDB by tmcneal on Monday September 12, 2011 @04:50PM (#2988850)
To be fair CouchDB is very explicit that to get any sort of performance, everything must be a view. "Ad-hoc queries" (i.e. queries that are written on the fly instead of uploaded as a view) are clearly stated as "for development only".

Where CouchDB really falls flat is for write-heavy applications. The default configuration in CouchDB is to not reindex a view until it has been read. When a read occurs, any new data in a view that was added since the last read must be re-indexed by executing the map/reduce functions on that data. If you're writing frequently to CouchDB but not reading a lot (as in a data warehouse) the first query you run is going to be extremely slow, since it will need to run map/reduce on a lot of new data. CouchDB doesn't distribute work to multiple nodes like Hadoop, and I've found even simple reduce functions to slow down re-indexing by a factor of 10. I think CouchDB has settings now to update the index on commit, or you could always run a cron job to regularly query the view and force a reindex, but it's still going to be slow.

BigCouch (https://cloudant.com/#!/solutions/bigcouch) might be a potential choice for data warehousing, since it advertises full compatibility with the CouchDB API but offers distributed map/reduce like Hadoop/Hive/etc. I haven't used it though.

Re:10gen raises $20m for MongoDB by alnayyir on Monday September 12, 2011 @06:05PM (#2989067)
Couch is definitely a lot more honest about their limitations than mongo or riak, but my experiences make me hesitant to recommend it to anyone not intimately familiar with those limitations.
Re:10gen raises $20m for MongoDB by jchrisa on Monday September 12, 2011 @07:54PM (#2989378)
This is part of what we are addressing with Couchbase Server, an autosharding rebalancing Couch fronted by memcached. For K/V read and write we measure microsecond latency.

We are currently optimizing the views for cluster access, but the design goal is to offer at least the query performance CouchDB offers on small datasets, even on very large clusters.

More info: http://blog.couchbase.com/couchbase-server-2-0-tour-and-demo

Re:10gen raises $20m for MongoDB by alnayyir on Monday September 12, 2011 @11:56PM (#2989877)
Is it going to have the same/similar multi-replication for enterprise only limitation that Riak has or is this going to be funded in some other way?
Re:10gen raises $20m for MongoDB by noodle on Tuesday September 13, 2011 @12:38AM (#2989960)
i would not use it in future projects, myself, because my company is currently using it in several products in several different ways, and it has been nothing but headaches, problems, etc..

the theory behind the thing is great. in reality, its buggy and not fun to work with.