News for hackers. Stuff that matters.
Re:MongoDB queries don’t always return all matching documents by Lazare on Wednesday June 08, 2016 @01:41AM (#11860259)
> Unless you have never written migrations in SQL before you would know that they are even scary and a big bunch of sql

I've written a ton of SQL migrations this year, none of them are scary or even really count as "big bunch of sql". If this describes your code, you should probably stop and rethink what you're doing, because that's bad practice.

The difference between, eg, merging your "first_name" and "last_name" fields into a single "name" field with Mongo or a SQL DB is basically that the SQL DB has better tooling so it'll be cleaner, faster, and more concise.

> Now in your migration script you have to hack SQL and language-based ORM commands

Also false. Nothing requires you to use the ORM commands if they don't make your life easier; literally last week I wrote a migration for some ORM based code using pure vanilla SQL because it was more appropriate. If you know what your ORM is doing, it's trivial. And if you don't, well, you have bigger problems.

> Actually they can be just as explicit and documented.

With a relational DB: Check the table schema.

With mongo: Dig through the app code and try and reverse engineer what fields the code is expecting. (Or pull up a few documents and see what fields actually exist, then ponder how you know whether you're looking at document that uses the latest schema, or maybe an outdated or dead document using an outdated schema. Also, how do you know what optional fields might be missing? Or what possible values the currently null fields might have? Good luck with that.)

> Mongo even has schemas now I think.

No. Mongo still lists flexible schemas as a feature, ie, they do not enforce any specific schema.

> SQL constraints are very limited and literally every application has additional constrains

Yeah, but what relational DBs are very good at is enforcing foreign key constraints. If your data is relational at all (and let's be real, very, very little world data isn't at least somewhat relational), you'll need it, and Mongo doesn't have it. (Well, Mongo doesn't even have foreign keys, but it lacks the equivalent feature for dealing with denormalized data too.) Managing to set some non-numeric characters in your phone number field when some of your app code contains a hard assumption it will be numeric is bad, yeah. But when you start screwing around with relations, foreign keys, failing to properly propagate changes to every copy of a denormalized data structure...oh man, you can spend days trying to untangle that mess. And it's a class of error that relational DBs don't have unless you misuse them badly.

In short, you seem to be suggesting that as long as you're blindly using an ORM, Mongo is almost as good and flexible as Postgres. Which...sure, okay? I guess?

Re:MongoDB queries don’t always return all matching documents by tremon on Wednesday June 08, 2016 @08:46AM (#11861833)
what relational DBs are very good at is enforcing foreign key constraints. If your data is relational at all [..]

Nitpick: the "relational" in "relational database" is not a reference to the concept of foreign keys, but to "relational algebra", the mathematical model for regular data structures that SQL is based on.

https://en.wikipedia.org/wiki/Relational_algebra

[dead]
Re:MongoDB queries don’t always return all matching documents by dkersten on Wednesday June 08, 2016 @07:18AM (#11861444)
written in a shitty language (SQL)

I disagree with this. After doing a bunch of work with SQLAlchemy and Hibernate over the years, I've developed a major dislike to ORM's as a replacement for SQL and would much rather write my queries in plain SQL. My currently favourite SQL library is HugSQL[1], which lets you write your queries in SQL in .sql files and HugSQL will load them and provide functions which you can call to invoke the query.

I find SQL quite solid for querying and updating data. Its what it was designed to do. Much more solid that the query API's I've seen in ORM's.

However, more importantly, when performance matters, it is very difficult to properly understand performance characteristics of complex ORM queries (I often find myself making the ORM generate raw SQL, which I then manually review to determine what its doing (eg what indexes it uses, etc)) and I find it much easier to optimise SQL than ORM calls.

My point isn't to badmouth ORMs. If you like them and they solve your problems, that is awesome. My point is that calling SQL a "shitty language" is purely subjective and your opinion and other people have different opinions. SQL is not an objectively shitty language. I think its a pretty good data query and manipulation language. Not perfect, but definitely not shitty.

And no, I'm not a DBA. I'm a programmer who hates having to mess around with databases. It is for this reason that I want things to just stay out of my way so I can get my work done with minimal touching of the database. I've recently found myself forced to do things I never wanted to do (1. change data capture on a MySQL database; 2. query and table optimisation to make a database perform and scale better) and for #2, my life would have been a lot easier if I could have worked with raw SQL.

[1] http://www.hugsql.org/

Re:MongoDB queries don’t always return all matching documents by orf on Wednesday June 08, 2016 @04:37AM (#11860851)
Tl:dr - you used a crappy ORM and now think all ORM libraries sick. So... use MongoDB... which is always accesses through an ORM...

All your comments come across as confused mate. Enforcing arbitrary constraints belongs in the business logic of your app, but dealing with consistency issues does not. Postgres handles row constraints just fine, you can enforce any arbitrary logic (i.e colX != colY and its not the first Tuesday of the month), combined with stuff like expression indexes, json type and actual schemas leaves me confused as to why you would advocate mongo so hard.

Just because you write your migrations in a language like JavaScript doesn't mean it's any better than pure SQL.

[dead]
Re:MongoDB queries don’t always return all matching documents by orf on Wednesday June 08, 2016 @08:38AM (#11861780)
You're replies in the comments are rude and borderline fanatical in tone. There are two classes of constraints: one being 'this string shouldn't be longer/shorter than x' or 'this value should only be between 1 and 10'. The other kind is 'on the first Tuesday of every month only during a full moon can this record be deleted, and only by a user with permission X'.

The first can easily be modelled in SQL, the second should be modelled in your app. The first are not arbitrary, the second are. Not hard to comprehend mate. Just because SQL can't model all constraints doesn't mean you should throw it out the window and reach for mongo.

Comments like 'none of you understand this because youre not programmers and therefore know nothing' is incredibly insulting. I'm of the opinion that you know nothing, and if you're happy knowing nothing with fellow mongodb users then so be it. Just keep your nonsense away from me thanks.

[dead]
Re:MongoDB queries don’t always return all matching documents by orf on Wednesday June 08, 2016 @12:21PM (#11863351)
> A number between 1 and 10 on Tuesday is only slightly different. You can say it is not arbitrary, whatever.

It's not arbitrary, because the constraints at least in Postgres are themselves constrained to validating a single row in a single table. Your app code can be truly arbitrary, you could trigger a device that posts a carrier pigeon to someone many miles away and waits for it to return with a result. That's arbitrary.

> Just in case you forgot the original claim, it was that if you use a SQL database, the constraints are in your database, but if you use mongo, the constraints are in your code.

Sorry, that's not what I was claiming. I was stating, correctly, that constraints relating to the data belong in the database. Constraints belonging to the business logic that powers your app belong in your app. You can be 100% sure that a constraint in your database will be respected, whereas the same can't be said of your apps code.

If you are given a specification that says "no username can be longer than 12 characters", that's what you add to your database constraints. A specification that says "No emails from temporary email providers are allowed" belong in your app code. Clear separation of concerns, that mongodb has no clue about. 'Just validate everything in your code, and if you get junk data in your database then I fucking hope your code can handle it'. Please.

> Therefore, saying that it is a great win that a SQL database can do some basic constraints checking

It can do pretty complex constraints. On the data. The more complex, completely arbitrary constraints belong in your app code.

> is a ridiculous argument because it hardly contributes any benefit due to the fact that even more complex constraints are in the code.

What. That makes no sense. So having 100% certainty that no column value in your database will violate any of the constraints you place on it 'hardly contributes any benefits' because... you have other, unrelated, business constraints in your app? What logic is this?

Re:MongoDB queries don’t always return all matching documents by chris_wot on Wednesday June 08, 2016 @07:19AM (#11861447)
What do you mean by consistency? Something tells me you aren't using it in the way most of us understand it...
[dead]
Re:MongoDB queries don’t always return all matching documents by chris_wot on Wednesday June 08, 2016 @07:55AM (#11861566)
Your code enforces consistency and detects when a transaction fails the consistency rule rolls back the changes to the state when the transaction started? That's some great code I'm sure, but a lot of effort.

The only limitation on an SQL check constraint is that you cannot use columns in other tables.

As has been pointed about by many of us slow learners, it's far easier to understand constraints in SQL than it often is to dive into a code base.

Re:MongoDB queries don’t always return all matching documents by orf on Wednesday June 08, 2016 @08:43AM (#11861817)
> IN MONGODB YOU HAVE TO WRITE YOUR CONSTRAINTS IN THE CODE AND THAT IS NOT THE CASE WITH SQL.

You moron, that's not a good thing.

Re:MongoDB queries don’t always return all matching documents by dang on Wednesday June 08, 2016 @03:17PM (#11864815)
Please don't respond to a bad comment by making the thread still worse.
Re:MongoDB queries don’t always return all matching documents by DoofusOfDeath on Wednesday June 08, 2016 @06:44AM (#11861299)
> No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language.

SQL is a DSL. It has pros and cons for data manipulation. For many combinations of (particular programmer) + (particular thing to get done) + (set of considerations), SQL turns out to be a better tool than a more general-purpose programming language. YMMV, obviously.

> Err no it isn't false. It is based on real-world experience, rather than "theory" about "what you can do". In the real world, people use ORMs and the ORM migration library, and then they find that the migration library doesn't do what they need it to do, so they have to use raw SQL. But then they run into the problem that because the ORM handles all of the generation of indexes and column names and so on, they have to figure out the correct naming of everything so that the ORM understands what you have done. This often results in failure because the ORM does not want you to tinker with the schema yourself.

It sounds like you're making a decent argument against manually modifying a database schema at the SQL level, behind the back of some ORM's which can't gracefully handle such surprises. But I don't think anyone is disagreeing with you about that.

> Which requires you to context switch from the code to the schema and piece together all of the relations based on foreign keys.

Many programmers can handle that complexity. Maybe the additional layer of indirection / complexity added by your ORM is the particular systems you deal with too hard for you to reason about?

[dead]
Re:MongoDB queries don’t always return all matching documents by chris_wot on Wednesday June 08, 2016 @07:31AM (#11861483)
You have a house with an array of rooms, and each room has a house with an array of rooms? Etc.

Unless that's a reference to a house in the room class, that is not in any way a good design. And if it is a reference, then you implement it like this:

House is a table

Room is a table that has a foreign key that references the House primary key.

Can't get much simpler than that!

Re:MongoDB queries don’t always return all matching documents by easuter on Wednesday June 08, 2016 @12:20PM (#11863331)
> No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language.

Your other comments already make you sound like a petulant child and a moron (or at the very least, someone with no actual real-world experience), but this just the cherry on the cake.

SQL is extremely powerful, and using it effectively doesn't take much more effort than learning another new mainstream programming language.

Anyway, enjoy your tortured use of shitty ORMs + Mongo for your toy projects; move aside and let the real programmers go on with their work.

Re:MongoDB queries don’t always return all matching documents by collyw on Wednesday June 08, 2016 @11:06AM (#11862705)
"No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language."

Aren't mongo queries written in a subset of JavaScript?

SQL syntax is a bit crappy, but it is still one of the easiest languages to comprehend what is going on when reading code written by someone else.

Re:MongoDB queries don’t always return all matching documents by matwood on Wednesday June 08, 2016 @10:32AM (#11862496)
> No the difference is one has to be written in a shitty language (SQL), whereas the other can be written in an actual programming language.

You must really not understand SQL. SQL is one of the easiest languages to work with sets of data.

Are you one of those people who pull back 100k records from the db in order to get a random 25 using a 'real' programming language?

Re:MongoDB queries don’t always return all matching documents by gerbilly on Wednesday June 08, 2016 @10:50AM (#11862603)
>written in a shitty language (SQL)

SQL is old but it isn't shitty.

Reasons:

1) It is based on a solid mathematical theory[1].

2) It is declarative. Replacing SQL with a procedural language would be a spaghetti code disaster.

[1] https://en.wikipedia.org/wiki/Relational_calculus