News for hackers. Stuff that matters.
Re:The Marketing Behind MongoDB by jdmichal on Tuesday August 29, 2017 @04:24PM (#15127320)
> For rapid prototyping of new projects, it could be entirely reasonable to trade off long term data-consistency benefits of having a well-defined schema in return for faster iteration times. But taking that schemaless prototype to production directly is decidedly unwise, so the final step in a prototyping process like this should involve a refactoring of your data model to introduce a schema/migrations system before that point.

I don't get it. If you're not in production anyway, you're not working with real data. And if you're not working with real data, you don't have to migrate anything. The entire problem just straight-up disappears. Just blow away your database and recreate it with the new schema. Modify your test data insertion scripts to match and load them.

Re:The Marketing Behind MongoDB by tomc1985 on Tuesday August 29, 2017 @05:24PM (#15127725)
Or alter your schema in the client. Quite easy to right-click and 'Add New Column' or ADD COLUMN or whatnot. I'm trying this with a new project and it is resulting in a data model that feels cleaner and hand-built.

One can simply capture the schema when they've found a good structure, and then use that as the starting point for a migration system (like ActiveRecord).

Re:The Marketing Behind MongoDB by lewisl9029 on Tuesday August 29, 2017 @04:37PM (#15127417)
The need to maintain an explicit schema and update it every time your data model changes is the point of friction here, not the migrations system. To be perfectly clear, I'm not suggesting maintaining a schema is onerous enough to outweigh the numerous other benefits having an explicit schema provides, even at the prototype stage, but when you're prototyping and don't need to worry about data migrations, having to maintain a schema is certainly not going to be _faster_ than not having to maintain a schema, as the original post claimed.
Re:The Marketing Behind MongoDB by wvenable on Tuesday August 29, 2017 @05:35PM (#15127789)
You always have to maintain an explicit schema -- it's just defined in your code instead of the database. And depending on your tooling, it's not easier/faster to maintain that schema in code vs. the database.

What is worse for schemaless databases is schema versioning. Your code will forever have to support every schema that has ever existed in production data. I have code to support schemaless data that was entered well over a decade ago. If I had simply used a relational database I would just have a single schema to deal with.

During development, schema changes are quite frictionless. It also provides extra safety and documentation that any friction is completely worth it.

Re:The Marketing Behind MongoDB by lewisl9029 on Tuesday August 29, 2017 @08:50PM (#15128866)
> You always have to maintain an explicit schema -- it's just defined in your code instead of the database.

This is normally known as an implicit schema. It's implicit precisely because you don't explicitly maintain a schema separately from your application code.

> During development, schema changes are quite frictionless. It also provides extra safety and documentation that any friction is completely worth it.

The second sentence contradicts the first. And I'm not disagreeing with the second. The friction may be worth it, but calling it _frictionless_ was my issue with this post and the original (which went even further by asserting the friction somehow accelerated development).

Re:The Marketing Behind MongoDB by wvenable on Tuesday August 29, 2017 @10:00PM (#15129176)
> It's implicit precisely because you don't explicitly maintain a schema separately from your application code.

But that's not completely true. There is plenty of code necessary for managing the schema and just the schema in a document database. Default values, relationships, validation, and managing previous version of the schema. I have plenty of code that just exists to handle documents in the "old" format.

> The second sentence contradicts the first.

Maybe I should have said almost frictionless instead of quite. And honestly, some things are less work with a schema (like creating a new defaulted boolean column) in an RDBMS than in a schemaless design. So the friction is relative.

> The friction may be worth it, but calling it _frictionless_ was my issue with this post and the original

I have a similar issue with calling a document-store _schemaless_. It's not schemaless, it has a schema. In fact, it has as many schemas as there are changes to the structure of the data. And this, in my opinion, is the biggest negative to that kind of design.

Re:The Marketing Behind MongoDB by lewisl9029 on Tuesday August 29, 2017 @10:38PM (#15129336)
I don't think we're really disagreeing on anything, just talking over each other a bit. Let me try to clarify myself:

> But that's not completely true. There is plenty of code necessary for managing the schema and just the schema in a document database. Default values, relationships, and managing previous version of the schema. I have plenty of code that just exists to handle documents in the "old" format.

This is only the case because you chose to _add_ an explicit schema on top of whatever schemaless DB you were using for maintainability reasons, which is certainly necessary if you're building a production app. However, in this case I was talking specifically about using schemaless DBs in the context of prototyping, and adding an explicit schema (that specifies things like default values, relationships, and migrations) is certainly not mandatory in the prototyping phase when you're not dealing with any real/past data or migrations.

MongoDB and other schemaless databases by default will happily accept whatever document you want to store in it without any care in the world, and this can be a desirable property for iterating as quickly as possible, but in production you definitely want to specify an explicit schema of some kind on top of Mongo as you have done, or just move off of Mongo altogether onto a proper relational database with a mandatory explicit schema.

As to which is the better long-term choice for a production app, I totally agree with everything you said. With schemaless databases, even if you add an explicit schema on top of it, there's always still the possibility to underspecify in your schema and re-introduce implicit data dependencies into your data model and application logic, which can lead to nightmarish bugs in production (I've experienced many instances of this first hand). It's much better to use a database designed around a mandatory explicit schema for relational data for the stronger data consistency guarantees they provide once you start handling real, persistent data.

Re:The Marketing Behind MongoDB by wvenable on Wednesday August 30, 2017 @12:39AM (#15129767)
When you're prototyping, you can be equally as sloppy with a relational database and make the same productivity gains. However, the sort of thing you describe doesn't even sound like a real prototype but rather something fairly trivial:

> by default will happily accept whatever document you want to store in it without any care in the world, and this can be a desirable property for iterating as quickly as possible

I fail to see how a database full of mismatched documents is valuable in prototyping. My own experience with prototyping in an RDBMS is just a constant evolution of the existing sample (sometimes real) data. Adding new columns is trivial, breaking up a single column to a one-to-many is a simple insert..select into a new table as one example. Similar transformation with a document-store involves writing a lot of throw away code.

But from another perspective entirely, I find actually designing the schema to be the best place start when I prototype an application. If I have the database design correct then designing the corresponding UI or API is almost trivial. Now obviously that's just one style of development but it's a no less valid one. And I don't have to do anything more to move to production.

Re:The Marketing Behind MongoDB by lewisl9029 on Wednesday August 30, 2017 @01:57AM (#15129982)
> I fail to see how a database full of mismatched documents is valuable in prototyping.

Again, as you yourself even brought up, just because you don't have an explicit schema doesn't mean your data needs to be schemaless. There can still be an implicit schema to your data that depends on the shape of the documents you store in your database. And in a prototype without real data (I'm defining real data here as data that can't be trivially discarded without consequence), the documents that actually end up in your database will have a uniform shape simply because your code only operates on the current version of that implicit schema, and older versions can simply be deleted.

> But from another perspective entirely, I find actually designing the schema to be the best place start when I prototype an application. If I have the database design correct then designing the corresponding UI or API is almost trivial. Now obviously that's just one style of development but it's a no less valid one. And I don't have to do anything more to move to production.

That approach is certainly valid. I just wanted to make it clear that using a schemaless database doesn't mean you can't do any schema design up front, it just affords you the ability to skip the overhead of updating and adhering to an explicit schema at every step in the evolution of your prototype. In order words, you let your code and your product needs drive the changes and growth in your schema, and only look towards solidifying the changes in your schema into an explicit document to adhere strictly to once you have more concrete insights into how your data model needs to look, driven by the data needs of a working prototype.

I feel I've stated my position as clearly as I could, so I'll leave it at that. Feel free to reply if you still take issue with anything I said, but I probably won't be responding.

Re:The Marketing Behind MongoDB by wvenable on Wednesday August 30, 2017 @11:17AM (#15132823)
I think you have stated it well. And I agree that there is overhead in creating a schema. I think, however, where we might disagree is on whether or not that overhead is anything more than trivial.

I think ultimately this is similar to the debate on static vs. dynamic typing.

Re:The Marketing Behind MongoDB by twic on Tuesday August 29, 2017 @04:52PM (#15127509)
If you're using an ORM, you can generate the schema automatically. With Hibernate, you set the hibernate.hbm2ddl.auto property [1]. If you set it to 'create' or 'update', it will alter the database schema so that it matches your objects, which is handy for prototyping, and terrible for production. If you set it to 'validate', it will check that the database schema matches your objects, and refuse to start if it doesn't, useless for prototyping but ideal for production.

[1] https://stackoverflow.com/questions/438146/hibernate-hbm2ddl...

Re:The Marketing Behind MongoDB by lewisl9029 on Tuesday August 29, 2017 @09:04PM (#15128933)
That's a neat feature, but as you may already know, ORMs come with their own baggage in the form of object-relational impedance mismatch. There's an entire rather-substantial Wikipedia article on it, so I won't go into it any further myself: https://en.wikipedia.org/wiki/Object-relational_impedance_mi...

The sibling thread also brings up similar issues with ORMs: https://news.ycombinator.com/item?id=15127058

Re:The Marketing Behind MongoDB by aryehof on Friday September 01, 2017 @09:32AM (#15148171)
ORMs are a solution to the problem of the impedance mismatch between an object model and its corresponding relational model. It's not that ORMs come with the problem.

I think it increasingly rare today to see real object domain models anyway - the original raison d'etre of ORMs. Instead it's common to see domain concepts modeled in the data layer and modified by logic (often limited to CRUD) in an application/service/controller layer. The problem with this of course is that as complexity increases, that app/service/controller layer increasingly approaches a big ball of mud.

Edit: grammar

Re:The Marketing Behind MongoDB by twic on Wednesday August 30, 2017 @04:44PM (#15135837)
Eh. I've used ORMs and SQL extensively for many years. I think the concerns over mismatch are routinely overstated.
Re:The Marketing Behind MongoDB by jimktrains2 on Tuesday August 29, 2017 @05:14PM (#15127650)
It's such a small amount of friction compared to the advantages of a good RDBMS.