News for hackers. Stuff that matters.
Re:What ORMs have taught me: just learn SQL (2014) by avmich on Monday December 18, 2017 @12:49AM (#15949702)
> A simple query builder will suffice and it will be much easier to debug and much less error prone than an ORM.

What's the difference? To me, an ORM is largely a query builder.

Re:What ORMs have taught me: just learn SQL (2014) by emilsedgh on Monday December 18, 2017 @03:45AM (#15950320)
ORM: You ask for something and you don't care _how_ its fetched.

Query Builder: You build a query, just not in SQL. So you can get around SQL's limitations (like composition)

Re:What ORMs have taught me: just learn SQL (2014) by andybak on Monday December 18, 2017 @04:16AM (#15950431)
> ORM: You ask for something and you don't care _how_ its fetched.

Try this:

> ORM: You ask for something and because you've researched and found a well-written, quality ORM you trust that it will create a sane query.

or possibly:

> ORM: You ask for something and you accept the tradeoffs vs hand-written queries but you're content that it's the correct tradeoff for your use-case.

There's plenty of places for bottlenecks to hide and premature optimization is the root of at least some evils.

Re:What ORMs have taught me: just learn SQL (2014) by jcadam on Monday December 18, 2017 @08:20AM (#15951393)
> ORM: You ask for something and because you've researched and found a well-written, quality ORM you trust that it will create a sane query.

Ha. More like:

ORM: You've just joined a project already using an ORM selected by an 'architect' that no longer works here. Everything is fine until you start testing your system with a database sufficiently populated with real-world data. You and the DBA spend the next next 6 months trying to get the damn ORM to generate performant SQL (you even call Oracle support, which promptly dispatches an "engineer" who tries to sell you on another $500k of crap you don't need that won't really address the problem). You eventually just start writing queries by hand wherever you find a bottleneck caused by the naive ORM, which you could have done at the outset, but no one who actually knew SQL well enough was on the team back then.

Re:What ORMs have taught me: just learn SQL (2014) by paulryanrogers on Monday December 18, 2017 @08:58AM (#15951597)
IME, it should never take more than a few hours to run down why the ORM made such a query. At a minimum most RDBMS's have query logging and can explain queries.

Ironically the last time I had a big ORM performance problem was Hibernate eager loading all joins. Diagnosing and fixing it didn't take more than an hour. (Though we did have a very experienced DBA at the time.) YMMV

Re:What ORMs have taught me: just learn SQL (2014) by emeraldd on Monday December 18, 2017 @08:58AM (#15951599)
You have to known when to break free of the ORM. They are great for graphs of CRUD operations and pretty nasty for much of anything else. You may come to a different conclusion depending on the project and dataset of course.
Re:What ORMs have taught me: just learn SQL (2014) by dorgo on Monday December 18, 2017 @04:09PM (#15955530)
And when you break free of the ORM prey to god that your ORM doesn't have a hidden cache somewhere. If it does then you spent a week tear out your hair until you figger out that the ORM' cache caused all the trouble.
Re:What ORMs have taught me: just learn SQL (2014) by collyw on Tuesday December 19, 2017 @06:00AM (#15959311)
This sounds like a distinction that you personally have made, that the wider community probably won't agree on.
Re:What ORMs have taught me: just learn SQL (2014) by petre on Monday December 18, 2017 @04:06AM (#15950398)
A query builder aids you at constructing queries, while an ORM builds queries for you, runs them and maps the output to objects. It's more sophisticated than a query builder.
Re:What ORMs have taught me: just learn SQL (2014) by zimpenfish on Monday December 18, 2017 @04:54AM (#15950570)
> A query builder aids you at constructing queries

... that you understand and can be sure are sensible.

> while an ORM builds queries for you

... that you have to hope are sensible.

That's one of the biggest flaws of the ORM for me - you have limited visibility of what it's doing to your DB.

Re:What ORMs have taught me: just learn SQL (2014) by lmm on Monday December 18, 2017 @11:08AM (#15952594)
Isn't that the same reasoning as "I don't like using high-level languages because they limit my visibility of what's running on my processor"? Do you write all your code in assembly?
Re:What ORMs have taught me: just learn SQL (2014) by _grep_ on Monday December 18, 2017 @11:40AM (#15952909)
I think the issue is that if you don't understand what a high-level language is doing under the hood, then you will constantly be surprised by side-effects. You shouldn't be using an ORM to substitute for your lack of understanding SQL; you should be using it to automate tasks.
Re:What ORMs have taught me: just learn SQL (2014) by scarface74 on Monday December 18, 2017 @05:57AM (#15950802)
How so? I can log the the sql being generated and look at the logs. Something I should be doing either way.
Re:What ORMs have taught me: just learn SQL (2014) by zimpenfish on Monday December 18, 2017 @07:13AM (#15951103)
If you have to log the generated SQL to understand what's happening, you're already behind the curve.

And then what do you do if the ORM is generating junk? If the answer is "use a querybuilder/handcrafted SQL for that one", what's the point of the ORM in the first place?

Re:What ORMs have taught me: just learn SQL (2014) by mgkimsal on Monday December 18, 2017 @07:41AM (#15951216)
> And then what do you do if the ORM is generating junk? If the answer is "use a querybuilder/handcrafted SQL for that one", what's the point of the ORM in the first place?

The point may be that 98% of the queries are just fine, and you've saved time vs writing by hand, and it may be easier to read/understand for the next people to have to touch the code.

Re:What ORMs have taught me: just learn SQL (2014) by zaphar on Monday December 18, 2017 @09:09AM (#15951661)
"saved time" at the least optional time to save time. ORM's are a maintenance burden. They obfuscate DB performance behind usually an enormous API surface.

Most software work is maintenance work. Optimizing for initial deployment is shortsighted.

Re:What ORMs have taught me: just learn SQL (2014) by scarface74 on Monday December 18, 2017 @09:41AM (#15951887)
What "enormous" API surface? It's not about "optimizing initial deployment". It's about "optimizing continuous deployment" and having "always releasable Software". I've worked for departments with 15 devs all working on the same codebase and we could release and rollback releases (A/B deployments) every week because we treated the database as a dumb data store. We had multiple branches at the same time, etc.

I've also worked at a companies where all of the logic was in ungodly stored procedures and getting anything released took months and we had a whole two weeks "hardening sprint" because we had to coordinate with the "database developers" and of course the entirety of the business logic was In stored procedures.

Most software being maintenance work is even more of a reason to optimize deployment. What good is unreleased software? The most important part of the business is releasing software. Most of my emphasis asan Architect is making sure we can release fast.

Re:What ORMs have taught me: just learn SQL (2014) by mgkimsal on Monday December 18, 2017 @10:47AM (#15952380)
I like the idea of stored procedures. I get some of the value they bring. However, the few times I've been on projects where sprocs where the primary focus of logic/truth/app... it was always a pain.

* The 'developers' weren't allowed to write the sprocs. We were at the biz meetings, but the DB guys were hardly ever there - their meetings were separate for some reason, but because devs were at the meetings, the devs were the ones who also were the face of the project. When the project was behind, we caught it in the neck, even if we were bottlenecked waiting for the DB team to write their logic and expose it to us.

* The DB team was always fewer people, juggling more projects, and other things like system uptime, maintenance, backups, etc.

* The sprocs were never part of version control or part of any source code that we could ever see as part of normal development. They typically weren't subject to any unit testing process.

The answer to all of this is mostly human management, structuring resources differently, coming up with different processes, etc. But any of those things would have changed the power dynamic, which seemed to be the purposes in those environments. Hey, I can write a stored procedure too - let me write them, and if a DB wants to 'review' them - or really, anyone on the team - please review and let's hammer it out and make it better. But roadblocking projects until the DB guys can 'get around' to writing our mission critical procs is just silly.

One other 'weird' division I saw a few places was this "developers can never have access to production systems - that violates XYZ" (a regulation, or some 'law' that was never produced, etc). I asked what the core issue was, and it was "you can't just have developers going on to production and just making changes on live systems - that's ... (against our policy, etc)". This was particularly challenging in a situation where a critical bug only happened on one production system, and we weren't allowed to replicate the database to another system, nor was anyone with any knowledge of the deployed code allowed to get on to the production system to even see if what was deployed was what we'd developed. But... this was still "our problem". Yet... the DBA in this case was "allowed" to get on the system and hand-write new triggers and sprocs to 'fix' our problem, all without documenting/testing his code, nor committing the code to any repo for us to even have visibility in to the data manipulation he was doing to 'fix' the problem we supposedly cause but couldn't investigate.

Again, I know this isn't a problem specifically with stored procedures. When sprocs have been promoted as the primary interface, however, it's usually been a political/power grab more than a technical benefit. And yes, again, I know there are technical advantages in some cases, but usually not enough to outweigh the drawbacks I've encountered.

Re:What ORMs have taught me: just learn SQL (2014) by collyw on Tuesday December 19, 2017 @06:47AM (#15959471)
Your complaints about stored procedures seem to be about the nature of the company where they were used rather than any inherent flaw with them.
Re:What ORMs have taught me: just learn SQL (2014) by mgkimsal on Tuesday December 19, 2017 @11:21AM (#15961248)
I believe I pointed out that I realize it's a human issue more than a technology one. It was easier for some people to get suckered in to the power dynamics being played because "DBA" was already seen as more of black-magic art sort of thing, and those guys were the "real wizards" and so forth, so whatever they say goes. It's not been everywhere I've ever worked where sprocs were used, but it seems to have been at the places where "stored procedures are law".
Re:What ORMs have taught me: just learn SQL (2014) by scarface74 on Wednesday December 20, 2017 @06:52PM (#15975034)
And it's because most developers can do sql but most "database developers" can't code. So of course they are protective over their code.
Re:What ORMs have taught me: just learn SQL (2014) by weberc2 on Monday December 18, 2017 @09:56AM (#15952010)
You haven't saved time compared to using a query builder though; in fact you've lost time due to the hit to readability and debugability.
Re:What ORMs have taught me: just learn SQL (2014) by collyw on Tuesday December 19, 2017 @06:49AM (#15959477)
How do you know this? You have no idea what bugs were encountered or how easy they were to fix.
Re:What ORMs have taught me: just learn SQL (2014) by weberc2 on Tuesday December 19, 2017 @05:39PM (#15965107)
We're conjecturing about hypothetical bugs. No one in this thread knows what bugs were encountered nor how easy they were to fix; they aren't real bugs. They're symbolic of the bugs that we encounter every day, which give us the experience on which we base our conjecture.
Re:What ORMs have taught me: just learn SQL (2014) by scarface74 on Monday December 18, 2017 @07:45AM (#15951238)
How often is the ORM generating junk? To say that the ORM is useless because occasionally it doesn't generate performing code (with EF and Linq more often than not it does generate performant code) could be applied to any high level construct. But I don't see people giving up modern languages to go back to writing everything in Assembly or even C.

Yes I optimize when my automated performance testing/stress testing, tells me I need to and may write handcrafted sql, but I've also handcrafted some classes in C back in the day when my old Windows Mobile app using the C# compact framework wasn't performing.

Re:What ORMs have taught me: just learn SQL (2014) by zimpenfish on Monday December 18, 2017 @08:28AM (#15951432)
> How often is the ORM generating junk?

From what I've seen (couple of home-grown ones, Class::DBI, ActiveRecord), "more often than you want".

(I'm willing to admit they may not be class-beating examples. :)

Re:What ORMs have taught me: just learn SQL (2014) by scarface74 on Monday December 18, 2017 @08:53AM (#15951572)
I meant a good ORM. But then again, my definition of a good ORM is an ORM with a language that treats queries as a first class citizen. EF with Linq doesn't really act feel like a separate framework since Linq and Expressions are built into the language.
Re:What ORMs have taught me: just learn SQL (2014) by zaphar on Monday December 18, 2017 @09:11AM (#15951672)
Those are more of a query framework than an ORM. If that's your bar then most ORMs are going to fail to meet it.
Re:What ORMs have taught me: just learn SQL (2014) by scarface74 on Monday December 18, 2017 @09:49AM (#15951944)
How is it a "query framework"? Linq works with objects and the Linq expression provider treats the Linq as data that translates the objects to Sql at runtime. That is by definition an Object Relational Mapping.

Of course most ORMs do fall short because most languages don't have the powerful concept of "code as data".

Re:What ORMs have taught me: just learn SQL (2014) by FLUX-YOU on Monday December 18, 2017 @08:11AM (#15951348)
>you're already behind the curve.

SQL syntax is extremely verbose, and compounds the more tables are involved in a query. You're not counting the time the ORM has saved you before having to resort to logging SQL.

Re:What ORMs have taught me: just learn SQL (2014) by akvadrako on Monday December 18, 2017 @05:38AM (#15950726)
SqlAlchemy Core vs SqlAlchemy ORM
Re:What ORMs have taught me: just learn SQL (2014) by cosarara97 on Monday December 18, 2017 @07:40AM (#15951213)
And doctrine DBAL vs doctrine ORM
Re:What ORMs have taught me: just learn SQL (2014) by Animats on Monday December 18, 2017 @03:51AM (#15950343)
ORMs are more useful as insert builders. Putting data from an object into the database is something of a boilerplate process. Queries vary with what you want to ask. Most of the time, you don't need all the fields, so filling up some object just because it has slots for everything is a waste of effort. Especially if it means references to multiple tables.
Re:What ORMs have taught me: just learn SQL (2014) by egeozcan on Monday December 18, 2017 @04:39AM (#15950510)
Good ORMs have Partial<T> and lazy loading of complex properties through proxies, which can be overridden with something like .With(x => x.ComplexProperty).

But of course, as the queries get more and more complex, the flexibility of the ORM syntax approaches the flexibility SQL. In the end, there are many situations one would rather just use SQL.

Re:What ORMs have taught me: just learn SQL (2014) by majewsky on Monday December 18, 2017 @04:43AM (#15950527)
I think the best ORMs are those that just leave out the "Relational" part entirely. So... "OM"?

For example, in Go, I use Gorp, which has a Select() function where you pass in the SELECT query string (plus bound values) and the target type, and it loads every result row into an object of that type. So you can have an arbitrarily complex SQL query as long as it starts with `SELECT one_table.* FROM`. That's a marvelous design.

And when you have to do a query that returns results from multiple tables? Guess what, you just use the normal SQL module from the standard library.

Re:What ORMs have taught me: just learn SQL (2014) by scarface74 on Monday December 18, 2017 @05:56AM (#15950797)
Why is it an either or? EF can do that but then you loose the benefits of a type safe language.
Re:What ORMs have taught me: just learn SQL (2014) by oceanswave on Monday December 18, 2017 @07:45AM (#15951236)
If type safety is so great, why isn’t sql statically type checked
Re:What ORMs have taught me: just learn SQL (2014) by majewsky on Monday December 18, 2017 @08:22AM (#15951402)
SQL is statically type-checked, at least in Postgres.

  postgres=# SELECT * FROM domains WHERE id = 'foo';
  ERROR:  invalid input syntax for integer: "foo"
  LINE 1: SELECT * FROM domains WHERE id = 'foo';
                                           ^
Re:What ORMs have taught me: just learn SQL (2014) by cpfohl on Monday December 18, 2017 @08:42AM (#15951510)
Not just PostGres. That's standard SQL. Columns have data types. Tables have schemata. Schemata are very strongly typed. Can't insert a 'full_name' column into a table without such a column.
Re:What ORMs have taught me: just learn SQL (2014) by da_chicken on Monday December 18, 2017 @09:02AM (#15951616)
Unless you're talking about SQLite, SQL RDBMSs are both static and strongly typed. The systems typically do allow some implicit type conversions, but type is critical to how a table works.
Re:What ORMs have taught me: just learn SQL (2014) by lmm on Monday December 18, 2017 @11:05AM (#15952558)
Because SQL predates a lot of modern techniques. A ground-up replacement written today probably would be statically type checked.
Re:What ORMs have taught me: just learn SQL (2014) by setr on Monday December 18, 2017 @02:13PM (#15954434)
What do you guys think the schema is?
Re:What ORMs have taught me: just learn SQL (2014) by lmm on Monday December 18, 2017 @02:54PM (#15954855)
Dynamic
Re:What ORMs have taught me: just learn SQL (2014) by int_19h on Monday December 18, 2017 @12:57AM (#15949733)
Is it a good one, though?
Re:What ORMs have taught me: just learn SQL (2014) by JamesBarney on Monday December 18, 2017 @01:08AM (#15949775)
Probably better than the one I could build by hand, with far better documentation, and greater ease in googling problems :)
Re:What ORMs have taught me: just learn SQL (2014) by failedxyz on Monday December 18, 2017 @01:21AM (#15949826)
while ORMs can do quite a bit of optimization, they're still general query builders and can't construct the optimal queries for your use case. if you don't know SQL, or you don't know what's going on behind the scenes, your ORM could be performing much larger queries than it really needs to, costing performance and time
Re:What ORMs have taught me: just learn SQL (2014) by gjjrfcbugxbhf on Monday December 18, 2017 @02:47AM (#15950133)
Most ORMs have way too write raw SQL. Furthermore someone using an ORM well will understand how their queries are mapping to SQL.

Your scenario happens with people that either don't know or don't care. They will write crappy queries with any tool.

Re:What ORMs have taught me: just learn SQL (2014) by collyw on Tuesday December 19, 2017 @06:53AM (#15959497)
Sounds about right.

I built a moderately complex application in Django at a previous workplace, using the ORM for most things, until the queries were too complex for the ORM.

Another guy connected to the same database and built some graphs using PHP and SQL. Guess who had to help him write the SQL when the queries got too complex for him? The ORM user.

Re:What ORMs have taught me: just learn SQL (2014) by stouset on Monday December 18, 2017 @02:43AM (#15950117)
But that's a completely orthogonal problem — the point made (way) above is that you need to understand SQL whether or not you use an ORM.
Re:What ORMs have taught me: just learn SQL (2014) by jbergens on Monday December 18, 2017 @02:08AM (#15949988)
Bascially all in-house built query builders I've seen has been much worse.