News for hackers. Stuff that matters.
Re:Problems with JPA/Hibernate by doctor_eval on Sunday April 11, 2021 @06:52PM (#26774288)
In my extensive experience managing teams using JPA ORMs including Hibernate and EclipseLink, you not only get to learn the unavoidable details of your target database’s SQL, but also the complex, non-obvious side effects - especially caching interactions and performance edge cases - of the ORM as well. You also get to learn two distinct but similar query languages (one of which you can’t use anywhere else but Java). You get to throw away all the semantics built into the database structure, including knowledge about indexes, and you need to duplicate almost everything from SQL into Java, in the form of complex annotations against entity objects that you often wouldn’t need with traditional SQL access patterns. And when something goes wrong, you have a large, complex framework, including an inscrutable caching layer, between your code and your data - which can make debugging JPA code very challenging.

In return you get reduced performance, overly complex and difficult to read SQL code generation, an inflexible entity model, and non optimal database access patterns.

JPA is an extremely complex framework whose benefits, in my opinion, outweigh the costs in only a small number of relatively simple use cases. And even in those simple use cases, in my experience it is far less performant (and double the LOC) than using SQL directly.

I’m not saying you can’t use JPA in large, complex applications, because you obviously can. But those applications are harder to write, much larger, less maintainable and less performant than the equivalent applications written with simpler frameworks or libraries.

I personally found MyBatis annotated interfaces to be the ideal way to map SQL to Java, but perhaps ApacheDB is better for smaller projects.

JPA is one of those fabulous engineering experiments that, sadly, didn’t work out. It breaks every rule in the engineering book in order to make SQL work like Java, but it doesn’t succeed, and I could never recommend it to anyone.

Re:Problems with JPA/Hibernate by cletus on Sunday April 11, 2021 @08:51PM (#26775056)
Man, you and I see eye to eye here.

I'm a firm believer in leaky abstractions [1]. I often found with JPA that I would be fighting the framework to get it to produce the SQL that I wanted. I encountered (then) ibatis and it was a breath of fresh air. It took care of much of the tedious column-to-Java mapping and didn't introduce another DSL.

As soon as anyone tells you "you don't need to learn X with our framework Y that sits on top of it", all that means is you have to learn X, Y and the X->Y and Y->X translations and all but the first is proprietary.

This was my core issue with GWT (Google Web Toolkit) too. Too many engineers who weren't interested in learning Javascript saw it as a way of avoiding learning Javascript.

It's been many years at this point since I've used it so this may have changed but I recall that query verification was a runtime error issue and there wasn't a good way to essentially check that all your queries were valid. I mean you could write this yourself but it seems like some CI/automation could've done a lot of the heavy-lifting.

IIRC the same went for integration testing and stub databases for reading and writing via ibatis/mybatis.

[1]: https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...

Re:Problems with JPA/Hibernate by doctor_eval on Sunday April 11, 2021 @11:03PM (#26775725)
I love the analogy to GWT and suffered it first hand - we believed the hype, that we didn’t need to learn JavaScript - but in the end we ended up needing to fully understand the whole stack in order to work through edge cases.

I don’t mind leaky abstractions but the killer with JPA was the cache. IIRC, mixing pure SQL with JPA was really tricky because you’d end up with cache inconsistencies. So the JPA abstraction was leaky but the the framework effectively assumed that it wasn’t, which lead to all kinds for workarounds.

Re:Problems with JPA/Hibernate by js8 on Monday April 12, 2021 @04:20AM (#26777078)
> As soon as anyone tells you "you don't need to learn X with our framework Y that sits on top of it", all that means is you have to learn X, Y and the X->Y and Y->X translations and all but the first is proprietary.

Exactly, except it's not just about language Y being proprietary. (I am using term "language" instead of "framework", because they are, in the broad sense, conceptual languages.)

It gives you a hint when the language Y is worth having in addition to language X - only if the language Y makes it significantly easier to model problem in your domain. Therefore, by definition, the language Y has to have some limitations that X doesn't. If Y is as a general language as X is (it can express same problems with similar effort), you end up doing busywork translating between X and Y.

And this problem plagues ORM frameworks of all sorts - it's not clear what is more easily expressed in the language of the ORM, which is not already expressed in the relational algebra (and SQL), and conversely, what are you giving up by using ORM. Although it happens in all sorts of DSLs.

Re:Problems with JPA/Hibernate by wrwatson on Sunday April 11, 2021 @08:28PM (#26774904)
I am called apon to investigate database performance problems. Teams using Hibernate sometimes send me the Hibernate query. This is not enough to understand the SQL which has been issued. I would also need to see all the entity objects, and maybe some configuration parameters. Without the SQL, which nobody can exactly predict, it is difficult to performance tune.

So they turn on logging, get the SQL, and email that unreadable mess. In most of the cases I have seen, the SQL is fetching much more from the database than what the code really needs.

The first step when optimising SQL is to only ask for data that you actually need. Hibernate, as I have seen it used, defaults to fetching too many columns. I cut the SQL down, and come up with a performant statement. The developer has the challenge of translating the performant SQL back into Hibernate.

Hibernate makes easy things easier and harder things harder.

Re:Problems with JPA/Hibernate by doctor_eval on Sunday April 11, 2021 @11:04PM (#26775730)
Meh, I think JPA makes easy things expensive and hard things really hard. :)
Re:Problems with JPA/Hibernate by de6u99er on Sunday April 11, 2021 @09:06PM (#26775148)
Having managed myself multiple projects using various JPA implementations I think JPA is great for relational database driven applications. Some of my projects contained 150+ tables. I always use a combination of JPA and JDBC. And everything derived from a base object that comes with create and update timestamps and users, plus a uuid based id field.

The trick is that while knowing what you are doing in regards of limitations and performance, you design the data model alongside cardinalities from user facing components. You'll end up with a highly normalized schema.

The nice thing is that you can create views later on and transform data into shapes that are more useful to others.

If you make the mistake to start with the data-model only without knowing how the data is being collected or used you will most likely spend lot's of time converting entity objects into other objects and the other way around. The worst thing I have seen in my career, was half of the business logic implemented in Java and the other half in PL-SQL.

JPA is great for large projects! I haven't had a single issue that we could not solve.

Re:Problems with JPA/Hibernate by doctor_eval on Sunday April 11, 2021 @11:11PM (#26775762)
JPA is OK (but only OK) if you don’t have anything other than JPA accessing the database but in our largest application (1,000+ tables) this was never going to be the case. We had plenty of experience with database design, that’s not the problem.

But we found even in simple applications with a dozen tables, the generated SQL was suboptimal. For example, to delete the children of a parent row resulted in a DELETE statement for each child row. I’m sure there are loads of good reasons why it did that, but it’a not something that you’d even consider in pure SQL.

The systems I build these days tend to put 100% of the database logic in the database using (you guessed it) plPGSQL, and leave Java (or actually Go these days) to do the non database transforms, networking etc. you could say that I replaced learning about JPA with learning more about the insane capabilities of my database.

I’m sure you (and half of HN) are gagging right now, but I know I’m not alone, and I’ll take 75% fewer LOC and 1-2 orders of magnitude performance improvement over JPA any day.

Re:Problems with JPA/Hibernate by tablespoon on Monday April 12, 2021 @01:21AM (#26776274)
> But we found even in simple applications with a dozen tables, the generated SQL was suboptimal. For example, to delete the children of a parent row resulted in a DELETE statement for each child row. I’m sure there are loads of good reasons why it did that, but it’a not something that you’d even consider in pure SQL.

IIRC, it does that if you use Lists instead of Sets.

https://dzone.com/articles/best-performance-practices-for-hi...

Re:Problems with JPA/Hibernate by Ygg2 on Monday April 12, 2021 @02:10AM (#26776448)
Sets in Hibernate suffer because you can store an object before it was updated and Hibernate will update it.

E.g. If say

    Person { 
       id: Integer, 
       name: String
    }
Let's say I have some entity that stores Persons as HashSet. If I add them before saving, their hash set is taken without Id.

Id can be automatically generated by Hibernate on save/create.

After save the Id now has a new value. I.e. its hash differs from when it was saved.

Boom. You have HashSet with Zombie entries. That are technically that you can't fetch actually.

Re:Problems with JPA/Hibernate by doctor_eval on Monday April 12, 2021 @01:54AM (#26776393)
Pretty sure I would have used a list, but making the Java interface ordered surely doesn’t change the semantics of the underlying relation?!
Re:Problems with JPA/Hibernate by tablespoon on Monday April 12, 2021 @11:29AM (#26780471)
> Pretty sure I would have used a list, but making the Java interface ordered surely doesn’t change the semantics of the underlying relation?!

Yeah, it seems a little bonkers on first glance and I haven't ever dug in to see if there was a justifiable reason or not.

IIRC, if you just need ordering, you could just use an SortedSet. I think technically lists are for when you need ordering and duplicates.

Re:Problems with JPA/Hibernate by doctor_eval on Monday April 12, 2021 @08:12PM (#26786687)
Yeah it is bonkers! Not just at first glance. The fact that the SQL integration for the child object behaves differently based on the kind of collection used in the parent is exactly the kind of unintuitive behaviour that drove me nuts.

There are so many rules to remember for something that is supposed to make SQL integration natural.

Re:Problems with JPA/Hibernate by imtringued on Monday April 12, 2021 @11:06AM (#26780198)
How do you get 75% fewer LOC without using JPA or Hibernate?

All I have to do to create a new table is define a class with some attributes and the corresponding SQL migration, I don't even have to annotate anything. To create a new row I just do new Table(attr1: expression1, attr2: expression2).save() and I am done.

Compare that to the usual mess of writing down the same column name four times with raw SQL (1. to reference the column, 2. to set the column value with a named parameter, 3. to set the named parameter, 4. declare the variable you wanted to save in the first place).

Of course, I'm not using Hibernate directly, that would be foolish. Instead I am simply using Gorm to wrap the awful parts of Hibernate (the irony).

Re:Problems with JPA/Hibernate by doctor_eval on Monday April 12, 2021 @07:53PM (#26786591)
So now you have code in Gorm generating DDL and JPA code? In terms of the compiled LOC, doesn’t this add even more code? Not to mention another dependency to maintain.

Anyway - I’m definitely not advocating for raw JDBC, there are lots of libraries you can use to reduce the complexity of the integration with the database, it’s just that JPA is not (IMO) one of them.

One thing I don’t understand about your comment it “the usual mess of writing down the column name four times with raw SQL” - when is this necessary? ’update table set column=? where key=?’ has no duplicate names. Can you give an example?

I’m intrigued by how your solution to JPA’s verbosity problem is to add another layer, while mine is to remove one!

Re:Problems with JPA/Hibernate by overtomanu on Tuesday April 13, 2021 @12:46AM (#26788215)
I think he is talking about upsert
Re:Problems with JPA/Hibernate by kimi on Monday April 12, 2021 @02:19AM (#26776485)
"Bad practice - if you hide the database, you may get something done quickly, but it's a bad idea. If your Java code expects to have a collection of one million objects as an array, it does not matter if they are lazily loaded or not - some code somewhere might want to iterate over them, and this will kill the process. You cannot really forget that there is a database somewhere, and you should not do it."

https://github.com/l3nz/ObjectiveSync

Re:Problems with JPA/Hibernate by doctor_eval on Monday April 12, 2021 @02:35AM (#26776572)
Great synopsis of JPA problems generally!
Re:Problems with JPA/Hibernate by edem on Monday April 12, 2021 @04:45AM (#26777204)
This comment is spot on. JPA/Hibernate is a very big leaky abstraction. That's why I ditched the whole thing and started to use things like jOOQ instead. In the end you must learn SQL to make sense of all of this anyway.
Re:Problems with JPA/Hibernate by eitland on Monday April 12, 2021 @01:24AM (#26776284)
Generally you should use the right tool for the job and that might be something else. This however:

> You get to throw away all the semantics built into the database structure, including knowledge about indexes,

suggests you either don't know JPA very well or your writing is a bit sloppy.

Please avoid making sweeping generalisations about tools that save hundreds (or al lot more) of hours of programmer tools just because you didn't get it.

Too often I see people going with a lesser alternative because such comments are scaring them away.

Re:Problems with JPA/Hibernate by sabellito on Monday April 12, 2021 @01:47AM (#26776365)
They made some concrete criticism based on their experience (hard to maintain, complex annotations, etc), but you in turn dismissed those with "you didn't get" and some borderline name-calling.

Which bit they didn't get?

Re:Problems with JPA/Hibernate by eitland on Monday April 12, 2021 @03:19AM (#26776778)
doctor_eval wrote:

>> You get to throw away all the semantics built into the database structure, including knowledge about indexes,

I wrote:

> suggests you either don't know JPA very well or your writing is a bit sloppy.

To suggest that JPA means "throw[ing] away all the semantics built into the database structure, including knowledge about indexes" indicates either that the person doesn't know what JPA is about: indexes should be used with JPA for all but the most trivial setups. Same goes for other general database knowledge.

You wrote:

> They made some concrete criticism based on their experience (hard to maintain, complex annotations, etc),

I just pointed out that it was either a misunderstanding or so sloppily written as to create misunderstandings.

> but you in turn dismissed those with "you didn't get" and some borderline name-calling. Which bit they didn't get?

The bit about JPA not standing in the way of using good database practices.

It is not name-calling. I point to the exact words doctor_eval uses. To be polite and also be sure to make room for error on my side I also offer the option of sloppy writing. Sloppy writing happens to all of us and I'll be happy to know if doctor_eval didn't mean it or if I've misread it.

If not however it is not smart to use big words to try to trash a super useful tool that save many of us lots of time and often increase quality.

Re:Problems with JPA/Hibernate by doctor_eval on Monday April 12, 2021 @07:17AM (#26778116)
I don’t think my point was sloppily written, but perhaps you misunderstood it.

JPA is conceptually incapable of automatically utilising the structure of the underlying database (despite the fact that the database structure is actually dynamic), so it makes up for it by requiring you to write these huge entity models which declare - and duplicate - that structure. You effectively “throw away” the database structure because you have to rewrite it in Java/JPA. I mentioned indexes but that’s just a side issue.

One consequence of this is that it makes refactoring much harder; I found that Java side of a JPA refactor was way more effort than the underlying SQL refactor, but that really shouldn’t be the case.

Re:Problems with JPA/Hibernate by eitland on Monday April 12, 2021 @11:46AM (#26780728)
I might start to get what you are hinting at. Still the

> JPA is conceptually incapable of automatically utilising the structure of the underlying database (despite the fact that the database structure is actually dynamic), so it makes up for it by requiring you to write these huge entity models which declare - and duplicate - that structure. You effectively “throw away” the database structure because you have to rewrite it in Java/JPA.

The way you wrote it: "you get to throw away" made it seem like you have to throw away your knowledge. (emphasis mine)

What you describe above is easy to read, but is of course not "throwing away" but duplication, which is of course an issue, but a totally different one.

> I mentioned indexes but that’s just a side issue.

Is indexes an issue with JPA or not? ;-)

You (or others) might wonder why I pick at your comment but I am so tired of seing people choosing inferior solutions because people are scaring them away from what would be perfect solutions for them.

Re:Problems with JPA/Hibernate by doctor_eval on Monday April 12, 2021 @07:37PM (#26786493)
It’s fine to pick at my comments, that’s what HN is for!

I accept that when I said “you get to throw away...” I was being a bit glib. I meant that we write the DDL, run it, and despite it being stored in the database as structure, that DDL is never used again by JPA and you have to rewrite it in annotations.

I thought lack of knowledge about indexes was an issue because I had a use case where the generated SQL was (very) sub optimal and thought it was due to an assumption made by JPA, but someone pointed out that maybe I used a Set instead of a List. I still think that’s stupid (it’s hardly fixing the “impedance mismatch” when which is the very purpose of JPA), but might not be index related. That said, knowledge about indexes does impact the design of handwritten queries and JPA doesn’t have that; JPA queries can be overly complex, and even pathological, so again I think the point holds up.

In my defence I was listing a lot of issues. I feel like I could write a book on the problems with JPA but all I had was this tiny box on my phone.

In terms of scaring people off, that is definitely my intent. There are lots of great ways to get Java talking to SQL, many linked to in this thread, and IMO the majority of developers would be better off choosing almost any alternative over JPA.

Re:Problems with JPA/Hibernate by eitland on Monday April 12, 2021 @10:15PM (#26787462)
I still think your experience is a bit on the extreme side but I'll still admit you've got me thinking: not because I've seen ORM causing much problems (I can use them just fine with indexes and custom SQL) but because maybe there is a better way.

Last I read up on MyBatis is probably >10 years ago and IIRC at that point it seemed like a manifestation of what the Rails guys teased us with:

  <situps>
    <up/>
    <down/>
  </situps>
but I'll admit:

- I've been wrong before

- things might have changed in 10 years

- I've always known there were cases for something else but maybe I should adjust my threshold

For anyone else who reads this, just be aware that many smart people are happily using JPA. :-)

Re:Problems with JPA/Hibernate by doctor_eval on Monday April 12, 2021 @01:50AM (#26776379)
I’m happy to scare people away from JPA! There are much better alternatives - such as MyBatis - that will make developers far more productive, and reduce LOC, heap size, and complexity.
Re:Problems with JPA/Hibernate by debarshri on Monday April 12, 2021 @04:07AM (#26777010)
Have you ever tried - ormlite-core[1]?

I recently found this obscure but quite stable library, kind of fell in love with it.

[1] https://github.com/j256/ormlite-core