News for hackers. Stuff that matters.
Re:Why I Migrated Away From MongoDB by taligent on Monday September 17, 2012 @04:36PM (#4534743)
This kind of rubbish really needs to stop. Just because you don't agree with or understand their choices does not mean that the majority of "NoSQL guys" are ignorant or uneducated.

Some of the biggest companies e.g. Twitter, Foursquare, Google, Amazon all rely on NoSQL.

The real issue I see is that by dismissing NoSQL as only for fools RDBMS developers are failing to see why they are popular to begin with. Take PostgreSQL for example and how difficult it is to shard/replicate compared to CouchDB or MongoDB. This is an area PostgreSQL should see as an opportunity for improvement.

But dismissing huge groups of people as uneducated just makes you seem uneducated.

Re:Why I Migrated Away From MongoDB by pessimizer on Monday September 17, 2012 @06:36PM (#4535276)
The majority of NoSQL guys are not Twitter, Foursquare, Google, or Amazon, and Twitter, Foursquare, Google, and Amazon all have plenty of people experienced enough in traditional RDBMS to tell them what things would probably be better in them.

Most NoSQL guys I meet have little to no experience in SQL. Plenty have lots.

Re:Why I Migrated Away From MongoDB by taligent on Monday September 17, 2012 @06:55PM (#4535330)
The plural of anecdote is not data. You know that right ? In my case every NoSQL developer I've met had a lot of experience in SQL as well.

But do you have ask why so called inexperienced users are choosing NoSQL in the first place ? Hint: it's because most RDBMS are ridiculously complex and inflexible.

Re:Why I Migrated Away From MongoDB by pessimizer on Monday September 17, 2012 @08:13PM (#4535576)
The plural of insulting everyone's intelligence who disagrees with you is not authority.

>But do you have ask why so called inexperienced users are choosing NoSQL in the first place? Hint: it's because most RDBMS are ridiculously complex and inflexible.

If we're going to have a cliche fight, this one is called having your cake and eating it too. Either inexperienced users are gravitating to NoSQL or they aren't.

Operating under the assumption that they are, I'd say it's both because they can interact with them without an impedance mismatch; also because they are flat, they are easy to visualize. Another reason might be because they don't have to put a lot of thought into their schema, which would involve new concepts that would take a little time to learn. The biggest reason in my eyes would be that they don't know how big a performance hit they're taking in a write-heavy environment.

In very read-heavy environments with wide heterogeneous queries that you would end up denormalizing in 18 different ways anyway? They could be doing it because they're smart and have done their research. I love NoSQL.

Re:Why I Migrated Away From MongoDB by taligent on Monday September 17, 2012 @10:47PM (#4536029)
I never meant to imply that inexperienced users were definitely moving to NoSQL. I would imagine that orders of magnitude more of them are still using MySQL due to its pervasiveness.

Only that if they were the current complexity of RDBMS would be a big factor. Accessing a database as a REST service like say CouchDB or having fluid schemas like MongoDB is much easier to handle than ER models.

Re:Why I Migrated Away From MongoDB by reinhardt on Monday September 17, 2012 @07:23PM (#4535420)
The last line sounds pretty familiar after replacing "NoSQL" with "PHP" and "RDBMS" with "decent languages".
Re:Why I Migrated Away From MongoDB by taligent on Monday September 17, 2012 @10:41PM (#4536016)
I completely agree. This whole NoSQL/RDBMS situation is programming language debate being played out over again.

And just like not every programming language works for every situation. Neither does every database.

Re:Why I Migrated Away From MongoDB by einhverfr on Tuesday September 18, 2012 @02:15AM (#4536638)
I agree. BTW, the most lucid description of when to use relational vs non-relational models is not all that new. You can read it here:

http://db.cs.berkeley.edu/papers/Informix/www.informix.com/i...

In this model, there is virtually no difference between the complex data/no query db's of the paper then and NoSQL today in terms of use case. Interesting that a one-time executive of Informix was able to present such a long-lasting discussion of db technologies.....

Re:Why I Migrated Away From MongoDB by gregjor on Monday September 17, 2012 @08:40PM (#4535662)
The plural of anecdote is not data.

OK...

every NoSQL developer I've met had a lot of experience in SQL

THAT is what data looks like, FTW.

Re:Why I Migrated Away From MongoDB by gaius on Monday September 17, 2012 @05:31PM (#4534996)
I think we did "sharding" with relational databases... Back in the 80s. Then we got fast hash joins and partitioning and it turns out that the disadvantages of sharding just aren't worth it. The NoSQL crew will figure this out too around 2030 :-p
Re:Why I Migrated Away From MongoDB by einhverfr on Monday September 17, 2012 @09:35PM (#4535833)
Be a little careful with this level of disdain.

Sharding may have been available in the 1980's, but what it lead to in some products is quite amazing. Consider Teradata's clustering ability which is sort of like sharding your database but without the disadvantages typically associated with it. Postgres-XC now offers something similar as a semi-fork of PostgreSQL.

Basically what we are talking about here is a two-tier database layer where storage and coordination are separated, and two phase commit is used between these two. Thus the coordination tier can enforce referential integrity between storage nodes if necessary and thus allow write-extensibility.

This isn't something without uses. For high-end, high-write-load databases, serving very large amounts of traffic (think airline reservations), this has been a typical approach for quite some time.

The fundamental problem though is that once you give up on local consistency over a given domain, you cannot have any guarantee of global consistency. The current relational approaches (Postgres-XC and Teradata) both enforce ACID compliance. BASE doesn't offer any consistency guarantee and therefore it is only good for throw-away data.

Re:Why I Migrated Away From MongoDB by gaius on Tuesday September 18, 2012 @12:42AM (#4536373)
Oh absolutely, but what you're talking about there, people do with CICS today, and that's even older than the 80s. CICS is a technology I have a lot of respect for.

But my point is - when I need to use something like that, I know that's what I'm using. I don't imagine that it's some new invention. Hell, a lot of what the NoSQL guys think they're inventing, IBM did back then too - IMS.

Re:Why I Migrated Away From MongoDB by einhverfr on Tuesday September 18, 2012 @02:03AM (#4536605)
"Those who don't understand Unix are condemned to reinvent it, poorly." – Henry Spencer
Re:Why I Migrated Away From MongoDB by taligent on Monday September 17, 2012 @05:40PM (#4535043)
I guess so will the likes of Facebook, Google, Twitter, eBay etc.

Must be amazing to know more about databases than all of the world's biggest companies put together.

Re:Why I Migrated Away From MongoDB by gaius on Monday September 17, 2012 @05:56PM (#4535120)
Why? The guys at FB or Google are just guys like you and me, don't believe the hype, they are not superheroes or gurus despite their much talked about interview process (the Google interviews I did were a walk in the park compared to GS, btw). Some will know more than me sure, but some'll know less.
Re:Why I Migrated Away From MongoDB by jeremyjh on Monday September 17, 2012 @06:59PM (#4535346)
Just guys like you or me except they build systems with billions of page views a month.
Re:Why I Migrated Away From MongoDB by gaius on Tuesday September 18, 2012 @12:49AM (#4536384)
At my last job, we averaged 60k pages/sec, on what the NoSQL guys would call a "legacy" database, and there was plenty of headroom. It's not rocket science, just engineering. Companies like Google love to weave a mystique around what they do, it's in their interests after all, to convince their investors that what they do is magic. But I'm from back in the day when we were the same way about millions of pages per month, then per week, then per day...
Re:Why I Migrated Away From MongoDB by einhverfr on Tuesday September 18, 2012 @02:23AM (#4536663)
On good hardware there were PostgreSQL instances running a billion queries a day back in the 9.0 days. Often these are actual accounting apps where reporting matters and so NoSQL would be a very poor fit. For example the French government uses it to distribute welfare program dispersements. The Wisconsin Courts also uses PostgreSQL at loads in the billion per day range. I know there are larger instances out there.

Now a days we are talking about thousands of concurrent users and up to 350k reads/sec on high end machines.

Re:Why I Migrated Away From MongoDB by gaius on Tuesday September 18, 2012 @02:33AM (#4536693)
Yes there is a lurking iceberg of things that aren't in the public eye like Facebook or Google, and that's where the really intense and interesting stuff is happening. I wonder what the guys at Visa make of all the hype around these websites, when they were doing these volumes all along...

At said 60k pages/sec job, you know who we looked up to for databases? Starbucks. Walmart. McDonalds. 'Cos we'd seen what they do, and anyone who thinks Twitter is impressive, well it'd blow their minds.

Re:Why I Migrated Away From MongoDB by einhverfr on Tuesday September 18, 2012 @02:52AM (#4536756)
Have you looked at Postgres-XC yet? It's Teradata-style clustering for PostgreSQL.....
Re:Why I Migrated Away From MongoDB by taligent on Monday September 17, 2012 @06:36PM (#4535277)
Those were only a few examples. I can keep listing companies that use sharding.

The idea that it is some 1980's approach to scaling databases is laughable.

Re:Why I Migrated Away From MongoDB by PommeDeTerre on Monday September 17, 2012 @09:07PM (#4535739)
You're right, it isn't a 1980s approach. I recall hearing about it in the mid 1970s, and even then it was probably originally devised in the 1960s.
Re:Why I Migrated Away From MongoDB by frio on Monday September 17, 2012 @06:03PM (#4535150)
This is a valid point. I dismissed NoSQL for years, and still (largely) do (I prefer to have a schema, documented data structures, deal with relational data etc.)... but I finally gave Mongo a go the other day, and am actually quite impressed. For a simple endpoint I can throw arbitrary persistent data in (rather than the filesystem), it's let me get a couple of proof-of-concept projects out the door much quicker.

Note that I'm not intending to comment on its scalability or how appropriate it is for other projects, just that I agree with you: dismissing it out of hand is hubris.