News for hackers. Stuff that matters.
Re:Velox: An open-source unified execution engine by polskibus on Thursday September 01, 2022 @09:02AM (#32675696)
I see this as a continued effort of middleware being rewritten in C++, Rust and Go to replace Java - seems like common wisdom "Java can be as fast as C" has finally been abolished as this situation progresses (Kubernetes and other newer cloud middleware written in Go instead of Java, etc.)
Re:Velox: An open-source unified execution engine by pjmlp on Thursday September 01, 2022 @10:58AM (#32677221)
Apparently you missed the part where it is integrated into Java libraries via JNI.

Kubernetes was originally written in Java and rewritten in Go after some Go advocates joined the team.

Docker was originally written in Python and rewritten in Go after some Go advocates joined the team.

Don't mix technical capabilities with people wanting to refresh their CVs.

Re:Velox: An open-source unified execution engine by boulos on Thursday September 01, 2022 @11:23AM (#32677711)
> Kubernetes was originally written in Java and rewritten in Go after some Go advocates joined the team.

Just to be clear, this didn't go very far. It's more like Brendan was willing to write Java, and a lot of other folks hate it. I don't think any of the core folks were Go advocates, just most of "us" were C/C++ people. Docker itself being in Go, meant it would lower a lot of friction, and besides we had great tooling for Go.

tl;dr: It was really "Should we (re)write this in C++? Eh, how about Go?".

Re:Velox: An open-source unified execution engine by pjmlp on Thursday September 01, 2022 @11:38AM (#32677952)
The history told at FOSDEM was a bit different, if I remember correctly.

And to come back to my point, if Docker or Kubernetes were invented today, I bet they would be using Rust or Zig instead of Go.

Re:Velox: An open-source unified execution engine by boulos on Friday September 02, 2022 @01:24AM (#32686912)
Maybe! I'd definitely argue for the Kubelet to be in Rust, if starting from scratch. The apiserver is less clear. But note that at Google it's unclear those would be written in Rust even today.
Re:Velox: An open-source unified execution engine by benreesman on Thursday September 01, 2022 @09:53AM (#32676226)
I broadly agree, though I think you might be under-counting the degree to which accumulated ecosystem value drives this phenomenon, especially amongst a group of languages not known for strong mutual interoperability.

Java is fucking fast, I imagine Cliff Click would like a word with someone arguing golang outperforms a well-tuned JVM. But the language isn’t aging the best and interop with non-JVM languages is, also not the best.

golang seems pretty optimized for polyglot SOA-type setups where a comparatively modest amount of existing code compared to say C++ isn’t a real drawback because you’re hitting that stuff over the network anyways.

Rust is probably the better language in C++ niches at this point when you don’t have a big C++ ecosystem investment and can do a more “greenfield”-type project. That can become a holy war and I’ll leave my position at: “there are good reasons to choose both”.

Re:Velox: An open-source unified execution engine by liuliu on Thursday September 01, 2022 @09:48AM (#32676153)
Containerization also helps the trend. Previously, you manage JVM runtime separately in your cluster and the deployment becomes simpler (just deploy the jars). Without containerization, maintaining separate runtime dependencies for compiled language is not as smooth. But containerization changed the equation. Now you deploy runtime dependencies along side your binary (if needed). It is actually a liability to deploy JVM runtime along side your jars now (actually, can you deploy java container images with JVM on the machine?)
Re:Velox: An open-source unified execution engine by Ingon on Thursday September 01, 2022 @09:59AM (#32676296)
For me, its mostly around tooling/dependency management. In go, you install the compiler and you are done - you can create a project, add dependencies (directly from github/git), and ship it as a self-standing binary.

In java (granted I haven't done java for a few years), you install the compiler/vm, and that is what you have. You have to decide if you are a maven or a gradle shop (and install these, or go bare lib/ mode), install and configure the said tools. When adding a dependency, you hope its on maven central, but sometimes its not, so random git repos are harder to try out/consume. Then eventually, you build your project and end up with a jar file, but you still have to manage its dependencies. You need the vm to run it, so you need to figure this out (jpackage also needs configuring in maven/gradle). You need your dependencies too, so you also need to figure this out (fatjar?).

Maybe things have gotten better in the recent years (and I'm happy to hear how), but my impression is still that the amount of dependency management you need to do with java far exceeds what you need to do in go.

Re:Velox: An open-source unified execution engine by pjmlp on Thursday September 01, 2022 @01:08PM (#32679559)
First of all you can deploy everything together, just like people ship their whole computers with containers.

Secondly anyone accessing Maven central directly is doing it wrong, the repo should be internal, validated by IT and legal, so for the consumers it doesn't matter how the JAR got there.

In a way it is ironic to see the whole containers/WebAssembly ecosystem redoing Java App Servers, 20 years later.

Re:Velox: An open-source unified execution engine by 62951413 on Thursday September 01, 2022 @12:02PM (#32678386)
The JVM trades memory for all kinds of optimizations so it's less about raw performance and more about memory allocation tricks (and related FFI issues). This is where golang at the moment has an edge over it. Assuming you can tolerate its primitive abstractions.

Once Loom&Co get merged, the JVM is going to be unrecognizable. It's also very possible that one of those Kotlin-native/GraalVM AOT projects goes mainstream. But right now it's a problem for high-performance systems.

Personally I regret this trend because Java/Golang are much easier for people like me than C++/Rust. I cannot imagine myself going back to manually managing memory. Browsing Big Data open source code will not be as educational anymore.

Re:Velox: An open-source unified execution engine by peaky_blinder on Thursday September 01, 2022 @12:21PM (#32678706)
You don't do manual memory management in Rust either. AFAIK (correct me if I'm wrong), even in C++ manual memory management is discouraged in favor of RAII.
Re:Velox: An open-source unified execution engine by pjmlp on Thursday September 01, 2022 @01:05PM (#32679494)
You surely do, first of all the borrow checker is only a compiler validation that you are writing the manual code correctly, and no one is magically writing Drop traits implementations for the user.

Likewise on the C++ side, someone has to write those constructor/destructor pairs, and there are ways to get RAII wrong.

Re:Velox: An open-source unified execution engine by cmrdporcupine on Thursday September 01, 2022 @03:13PM (#32681550)
Seriously, no, re: Rust.

Yes, you have to think about ownership. Because of single owner. But in general, not alloc/free. I work in Rust (after doing C++ for the 10ish years prior) full time and a couple nights ago was honestly the first time I had to really think about this (I had to do a 'forget' because I was passing a ptr to a vector back through from WebAssembly to our Rust-based runtime, and then clean it up back there instead of having Rust let it fall out of scope and free when the stack frame exited).

Think of Rust as a world where pretty much everything is inside a std::unique_ptr.

Most developers used to a GC language will have little problem working in Rust once they understand the single ownership model.

Re:Velox: An open-source unified execution engine by pjmlp on Thursday September 01, 2022 @03:40PM (#32681927)
Try to write a native GUI application in Rust, or async code, and you will see how little you have to think about it.
Re:Velox: An open-source unified execution engine by cmrdporcupine on Thursday September 01, 2022 @03:48PM (#32682052)
Yes, GUI work is a pain because of the way event loops and ownership in existing UI toolkits work, they're generally not designed for this. But Arc<Mutex< is likely your friend here.

Async can be a pain, but you learn the ways. I work in a codebase with quite a bit of it.

There are appropriate and inappropriate places to apply Rust.

Re:Velox: An open-source unified execution engine by pjmlp on Thursday September 01, 2022 @05:27PM (#32683490)
It surely is my friend, and I will need to manually call clone() and borrow().
Re:Velox: An open-source unified execution engine by hobofan on Thursday September 01, 2022 @02:33PM (#32680924)
> no one is magically writing Drop traits implementations for the user.

Yes, the compiler is - for almost all structs. I've been working with Rust since around 1.0 and I can count the number of times I manually had to write Drop implementations with my hands. Unless you are writing lower-level parts of the stack (which you rarely need to, since for many of those there are good crates already available) where you are responsible for resources that need a custom Drop implementation, the auto Drop implementation is good enough.

Re:Velox: An open-source unified execution engine by pjmlp on Thursday September 01, 2022 @03:07PM (#32681454)
Trivial types don't count, their Drop implementation is basically do nothing.

And for Rust standard library, where the Drop actually does something, someone else wrote the implementation for you.

Re:Velox: An open-source unified execution engine by cmrdporcupine on Thursday September 01, 2022 @09:55AM (#32676243)
Maybe, except this isn't "middleware?"

It looks to me like the kind of librar(ies) that would normally be used as an internal aspect of a database or data analytics tools. Data containers / structures, data manipulation operators, and facilities for moving them around.

What this is is part of nice trend to open source some of the fundamental R&D that is happening inside the BigCorps.

Also, this kind of thing (and, indeed middleware as well) has been done in C++ by default inside Google (and probably Facebook as well) since forever.

Re:Velox: An open-source unified execution engine by zaphar on Thursday September 01, 2022 @09:30AM (#32675966)
I think it's a little more nuanced than that. Java was fast enough combined with security and safety guarantees to make it worth it in a total calculation. But Go and Rust are both faster with at least as good if not better safety guarantees so you are seeing a move away. Language choice is never wholly about speed. Java filled a niche of fast enough with certain guarantees and affordances to allow it to fill certain roles.

C++ has gained a lot of affordances as well to improves safety as long as your can enforce their usage in the product so it's starting to eat into Java's market share as well. But I think long term Rust and Go will absorb more of what you would have done in Java than C++ will.