I have tied to use automatic migrations and I do not see the point. Either your diff is trivial and it is trivial to write a migration or your diff is complex and you have to care about the lock levels of schema changes, the time complexity of changes and that the migration transforms data correctly. The code generating the automatic migrations needs knowledge about the structure of the production data.

exactly

Lack of adequate support for versioning your schema and in general bad support for anything versioning with RDBMS is one of the main dislikes I have for SQL. I find SQL to be a beautifully expressive language when you get the hang of it, but no RDBMS that I know of has been able to adequately tackle the versioning problem at both the schema and the script level. Everyone's schema evolves over time, why is it so difficult for the RDBMS to try to capture the idea of database evolutions? This is the single biggest selling point for me in ORM land, because with regular (read: non-SQL) code you have all the typical version control mechanisms to help you manage changes to the schema. It's still not ideal, but a lot better than trying to manage this kind of thing in any of the tools that mainstream RDBMS have.

Yes I agree. I ended up making a small DSL to describe tables. Then another program to translate that in to a script that checks everything through meta data in the target db (atm it does mysql). I check the table structure files into version control. If I wanted an old version pull that checkout and run the generator script. I end up with a pure SQL script that creates a blank db with bootstrap data or asserts all the columns/indexes exist for that version of the code.

There is another small dsl for bootstrap data, and I just check in all stored procs like normal code.

What this doesn't give you is good change column type semantics but any other tool I've ever used when this happens, can't be trusted anyway you have manage it yourself. This type of change is pretty infrequent though.

For me it works better to assert the schema I want now. Drop columns arn't done due to similar thorny issues that usually a human has to deal with. Or if its simple I put those in a cleanup proc that runs as the last part of the script.

Using normal code to describe structure I guess is similar its just you're fighting the language with meta data extensions, and also whatever you do isn't language agnostic. You're also at the mercy of whatever magic is going to happen when you change schema with that lib.

This is a big thing for me. Most ORMs I’ve used have decent migration tools which is a huge benefit for managing your schema. We enforce changing Schemas via migrations as opposed to any sort of auto-migrations some packages offer. Makes life a ton easier being able to put these changes into a build pipeline.

Migrations (or evolutions, depending on the language you are using) are a joy when implemented properly. Alembic and Play Evolutions are two frameworks that I have dealt with specifically that are pretty great and hard to go back from.

I'm very comfortable with SQL, but I also like the static typing and compile-time safety that linq brings.

Automatic migrations I like less. In particular, Entity Framework and Entity Framework Core migrations.

Firstly, it seems that the second you stray from the most basic of scenarios, thry generate incorrect DDL.

Secondly, I'm just not a fan of 'magic' code gen.

Thirdly, every entity/model change a dev makes does not need to equate to a schema migration - that's how you end up quickly accumulating hundreds of schema migrations.

When I have to use MS SQL Server, the approach I favour is to use CQRS, using EF Core for the majority of cases, then using Dapper when I need more control. And for migrations I like DbUp, because it provides me with full control over when it's time to create a migration file, and what better DSL for migrations than... SQL.

Damn that's a lot of enterprise level words and acronyms to just "store some data". Lol not saying you wrong... just don't like this part of our industry. Is it too late to go back to Turbo Pascal (record-files) ? :)

Not sure I really mentioned much 'enterprise-level', beyond SQL Server?

I guess I'm making it sound more complicated than it really is then, because this approach is actually a lot simpler than the 'N-tier' abstraction-hell of yesteryear.

Jimmy Bogard has a simple example here: https://github.com/jbogard/ContosoUniversityDotNetCore

Some code can probably explain it better than me ;)

Not to mention code reuse. Say for instance we have one query that finds eligible bachelors near me and another that finds newly eligible bachelors. We need two queries but they both rely on the same underlying domain concept of "eligible bachelors". In entity framework it's easy to save that as an expression that can be reused in multiple places. It's hard to do that in sql in a maintainable way.

Not to mention refactoring issues. If I decide to change a datetime field called time to split it into a datetime and offset that change is far safer in c# than it is SQL.

> Say for instance we have one query that finds eligible bachelors near me and another that finds newly eligible bachelors. We need two queries but they both rely on the same underlying domain concept of "eligible bachelors". [...] It's hard to do that in sql in a maintainable way.

Seems a textbook case of using a view for sharing query logic.

> If I decide to change a datetime field called time to split it into a datetime and offset that change is far safer in c# than it is SQL.

If you can do it in C# instead of SQL, you must be talking about a change in the interface used by an app (so, with good DB design, a view, not a base table). But this is very safe in SQL, you are changing a view definition and, if its a read/write view, insert and update triggers.

>Seems a textbook case of using a view for sharing query logic.

Views are just terrible outside of data analyst style work. Unmaintainable, restricted to SQL structures, require absurd hacks or custom dlls, not properly source controlled, difficult to perform performance analysis on, hard to update.

You can't combine views easily, you can't cache results, you don't get static type checking.

> Unmaintainable,

Views are not "unmaintainable".

> restricted to SQL structures,

I'm not sure how that's a problem, since SQL is a fully-general data language.

> require absurd hacks or custom dlls,

Views require neither hacks nor "custom DLLs" in any RDBMS I am aware of. Can you provide a concrete example?

> not properly source controlled,

There's no reason DB schemas (including, but not limited to, view definitions) can't be source controlled.

> You can't combine views easily

Yes you can, whether by "combine" you mean as joined source relations, or as criteria that need to be combined to slice and dice the same source data. That's, like, playing right to the strength of SQL and RDBMS systems.

> you can't cache results,

Well, not generally incrementally or in something like a recently-used cache; a number of RDBMS's do support materialized views, though.

I don't like to say "you're doing it wrong", but TBH it sounds like you are.

> restricted to SQL structures

So is your ORM; it just adds a layer of abstraction

> require absurd hacks or custom dlls

Literally no idea what you're doing that views require anything out of the ordinary, or indeed "custom DLLs"

> not properly source controlled

Eh? You can store your view DDL files in source control just fine. I mean, they're just SQL files; why couldn't you?

> difficult to perform performance analysis on

I don't see how ORMs offer any advantage here - if anything, SQL has the edge here, since your typical SQL GUI will draw pretty graphs to help visualise EXPLAIN, and some (e.g. SSMS) will even suggest missing indexes

> hard to update

It's just SQL, and for views typically rather basic SQL at that.

I use ORMs on 99% of the projects I work on, sometimes heavyweights like Entity Framework Core or Marten (which I love, BTW), and sometimes more lightweight like Dapper - but I know when to use the right tool for the job, and sometimes that's SQL views. I just don't get any of your arguments against them.

What SQL GUIs do you use that draws graphs for EXPLAIN statements? The ones I've used for OSX (PopSQL, Sequel Pro, TablePlus) hasn't had that feature. Haven't used MySQL Workbench in a few years, so not sure about that one.

pgadmin? https://www.pgadmin.org/screenshots/#9

I mentioned SQL Server Management Studio (SSMS), which is a great GUI for SQL Server. There is also pgAdmin (both 3.x and 4.x have this feature), and it's also available in Azure Data Studio (for SQL Server; it's coming soon for the Postgres version).

At least for MS SQL, code is fully source controllable, I have it on Git and we have all automated deploys and niceties.

Views can be a problem from a performance side, Oracle has materialized views but MS SQL lags behind (I think Postgre has them too).

> It's hard to do that in sql in a maintainable way

<cough>Views</cough>

I have spent enough of my life untangling a mess of views that create performance problems to doubt that views are maintainable.

Also, there is usually no good solution for keeping the database in sync with code releases. With the result that database+code change synchronization tends to be a source of deployment complexity.

Keeping the logic in one place is better than the alternative. And of the two, the tools to manage source control are better.

>I have spent enough of my life untangling a mess of views that create performance problems to doubt that views are maintainable.

Just like concepts like inheritance can be a burden when overused, views can be overused as well. Like inheritance, having views go too many levels deep generally winds up causing more harm than good.

Used correctly though, views are able to capture and centralize concepts in a way that are reusable, can be more easily tuneable, and make maintenance easier in the long run. Queries can be simpler and encompass less responsibility as the common logic is implemented by the view, and the logic that is specific to that query is more readily apparent.

IMHO thinking of the DB as just a collection of tables misses out on a key db feature.

>Also, there is usually no good solution for keeping the database in sync with code releases. With the result that database+code change synchronization tends to be a source of deployment complexity.

I disagree that there's no good solution, but I understand that it's not as "out of the box" as many ORMs offer. That being said, writing SQL scripts that coordinate schema changes is not hard, and any CI/CD pipeline that can run commandline utilities can run a sql script against a database.

At my work, we use DbUp (https://dbup.github.io/) and are able to easily write small, isolated SQL scripts that correspond with code changes and get committed to source control, run all of those scripts with our CI/CD pipeline across multiple sharded databases, and do so quite seamlessly. We don't have any questions about migrations or what a tool did, as running the SQL script individually is the same as it runs on deployment. Coordinating db changes on developer machines is a breeze as well.

>And of the two, the tools to manage source control are better.

All of our sql scripts are committed into source control. It's not any easier or harder than looking at a code file, an html file, or a config file to get to the logic your working with.

In the common setup where there is a central database shared by developers, you really can't keep the database in sync with all of the code. Furthermore the manual nature of keeping things in sync means that there will be errors. Hopefully those are caught in staging, but I've seen this not always work.

We use liquibase https://www.liquibase.org/ for tracking, managing and applying any schema changes. The liquibase changelogs are part of our project releases.

> Supports XML, YAML, JSON and SQL formats

XML is the default on the page, but wow the SQL is _so_ much shorter and more readable than the others! To the extent that I wonder what's the point, the obvious is machine readabaility, but one not three, and is SQL really so machine-unreadable that it's worth the extra depth and complexity?

With Liquibase, when you write changesets using SQL, you have to write the rollback code yourself. When you write them with Liquibase's built-in operations in (for example) XML, the rollback is derived automatically.

I write SQL almost daily, but for the things that can easily be expressed with Liquibase's operations, I can write them faster and with fewer errors in XML, so long as I'm using a text editor with XML schema support (hence: autocompletion and validation as-you-type). PyCharm works well for this.

I much prefer having an vs sql project side by side with the ef models. Keep the schema up to date in there. It can generate migration scripts by comparing the dB you want to target. Then just generate ef models from the dB.

I've been wanting to build static typing for raw sql atop https://github.com/uber/queryparser