News for hackers. Stuff that matters.

A comment on Building a faster hash table for high performance SQL joins.

Re:Building a faster hash table for high performance SQL joins by pixelpoet on Wednesday December 20, 2023 @01:00PM (#38711564)
Serious question, if performance is the lynchpin, why write it in Java?

Especially considering they use unsafe "heavily", for big joins they could easily just call out to some native code if the surrounding code reaaaaally must be Java (again, why?). It's the worst of both worlds using unsafe Java: you don't get native speed, there's loads of memory overhead from everything being an Object (besides the rest of the VM stuff), and get to "enjoy" GC pauses in the middle of your hot loops, and with fewer safety guarantees than something like Rust.

[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by lukax on Wednesday December 20, 2023 @01:18PM (#38711787)
They started with Java and tried to use as little GC as possible and started writing hot loops in C, C++ and Rust. Logic is in Java, other stuff is mostly native.

https://questdb.io/blog/leveraging-rust-in-our-high-performa...

[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by bluestreak on Wednesday December 20, 2023 @01:16PM (#38711764)
this is THE number one question being asked about QuestDB :) There is a thread that might help https://news.ycombinator.com/item?id=37557880
[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by gavinray on Wednesday December 20, 2023 @01:36PM (#38712020)
In the most non-inflammatory way possible: I am not sure I'm convinced of the performance benefits of crossing a JNI boundary to invoke a method in another language, versus letting the JVM JIT and optimize native JVM code.

Would be interesting to see benchmarks here, do you know if there are any public ones available, specific to QuestDB's interop code?

[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by bluestreak on Wednesday December 20, 2023 @02:13PM (#38712416)
In certain situations, crossing the JNI boundary can be advantageous. When data resides in "native" memory, outside the Java heap, the coordination of concurrent execution logic can be handled in Java, while the actual execution occurs in C++ or Rust. In this context, the negligible penalty for crossing the JNI boundary once per 10 million rows pales in comparison to the substantial benefits achievable through C++ optimization or CPU-specific assembly.
[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by pixelpoet on Wednesday December 20, 2023 @01:22PM (#38711839)
I see... but that seems a little weak considering it's a funded product, the first adjective they use to describe it is "fast", and good old C++ would totally slay it. The author has a C++ background, maybe he could spend an afternoon trying that?

I couldn't even try to count the number of great posts I've read about fast hash tables from e.g. Paul Khuong alone...

[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by dimgl on Wednesday December 20, 2023 @02:30PM (#38712586)
Interestingly I recently saw an MQ broker written in Java and it seemed to have some pretty impressive performance. I'm not a huge fan of Java's DX but I have to admit, that's some serious performance. Can't remember the name but it was on the homepage this week I believe.
[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by winrid on Wednesday December 20, 2023 @06:43PM (#38715155)
Isn't javas DX pretty great? I mean not the huge XML heavy frameworks but Java itself and the IDEs work well...
[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by dimgl on Thursday December 21, 2023 @02:32PM (#38725556)
Maybe it's inexperience but I never want to deal with Maven/Gradle again.
[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by winrid on Friday December 22, 2023 @01:29AM (#38731536)
Gradle's upgrades can be pretty terrible. I don't know why they keep changing it.
[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by toasted-subs on Wednesday December 20, 2023 @02:16PM (#38712446)
Yeah, I genuinely don't understand why so many people reach for Java for everything.
[ Parent | on HN ]
Re:Building a faster hash table for high performance SQL joins by marginalia_nu on Wednesday December 20, 2023 @02:32PM (#38712603)
It's a very well rounded language is why, and whatever papercuts exist are either on the way out in newer/future java versions, or made up for by the tooling ecosystem.
[ Parent | on HN ]