News for hackers. Stuff that matters.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by jonahx on Sunday March 03, 2024 @12:26AM (#39578661)
Nice post. Interesting that the fastest Java beats the fastest Go, though they are close:

    AY fastest Go version 2.90s 36.2
    TW fastest Java version 0.953s 110
I would have expected Go to win. That JVM works pretty good...
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by Mawr on Sunday March 03, 2024 @01:49AM (#39579018)
That's only a valid comparison if the "fastest Java" and "fastest Go" implementations are either the same or at the limit of what each language allows.

The more interesting comparison anyway is performance of the straightforward, idiomatic code, since that's what we all write 99% of the time.

Here's the key insight from the article: "Processing the input file in parallel provides a huge win over r1, taking the time from 1 minute 45 seconds to 24.3 seconds. For comparison, the previous “optimised non-parallel” version, solution 7, took 25.8 seconds. So for this case, parallelisation is a bit faster than optimisation – and quite a bit simpler."

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by dsff3f3f3f on Sunday March 03, 2024 @12:29AM (#39578675)
The Java version does additional optimizations that his Go version doesn't do and he mentions that at the end of the post. The Java version is really optimized and is an interesting read.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by avinassh on Sunday March 03, 2024 @12:38AM (#39578714)
> The Java version is really optimized and is an interesting read.

is there any similar blog post on the Java optimisations?

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by lucianbr on Sunday March 03, 2024 @03:33AM (#39579422)
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by cempaka on Sunday March 03, 2024 @01:22AM (#39578911)
Cliff Click did a walkthrough of his solution on YouTube: https://youtu.be/NJNIbgV6j-Y?si=Wj97f-Imw5nfIzF7
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by o11c on Sunday March 03, 2024 @12:49AM (#39578758)
https://news.ycombinator.com/item?id=39467885 but it looks like there have been improvements since.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by dsff3f3f3f on Sunday March 03, 2024 @12:44AM (#39578728)
Not that I know of. I just looked at the code and the commit history but a more in depth article would certainly be interesting.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by neonsunset on Sunday March 03, 2024 @02:11AM (#39579099)
The more accurate statement would be is Go implementatation is incapable of accessing optimizations that exist in Java and then Java is incapable of optimizations performed by C# and C++ implementations.

See https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-...

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by dsff3f3f3f on Sunday March 03, 2024 @02:37AM (#39579213)
Go is perfectly capable of all of the additional optimizations that are in the fastest Java implementation that is linked in the article.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by neonsunset on Sunday March 03, 2024 @05:20AM (#39579891)
Until some time ago Go did not even have inlining profitability logic in the compiler and could only inline leaf functions, something which is worse than JVM implementations could do in...2005 or so I think?

Are you sure?

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by AtlasBarfed on Sunday March 03, 2024 @03:26AM (#39579398)
Well the Go guy probably didn't read it because "java doesn't interest him"
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by threatofrain on Sunday March 03, 2024 @01:47AM (#39579013)
I don't think Go has ever demonstrated that it deserved to be thought of as casually faster than Java.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by Comma2976 on Sunday March 03, 2024 @03:35AM (#39579435)
https://github.com/attractivechaos/plb2/blob/master/README.m...

Synthetic benchmarks aside, I think as far as average (spring boots of the world) code goes, Go beats Java almost every time, often in less lines than the usual pom.xml

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by iraqmtpizza on Sunday March 03, 2024 @03:51AM (#39579493)
Anyone judging Java performance based on starting up a JVM, running for 30 seconds, and shutting down is brain damaged. I'm 99 percent certain this benchmark doesn't use Epsilon GC either lol

>not using GraalVM for Java

LOL k

>no implementations use multithreading

ah, the cherry on top. this benchmark is literally perfectly useless

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by Comma2976 on Sunday March 03, 2024 @05:39AM (#39579974)
And yet, people will refer to some other synthetic benchmark when arguing that the average Java project, which doesn't disable GC nor uses unsafe reads et al, is comparable.

Can't have and eat it too

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by iraqmtpizza on Sunday March 03, 2024 @08:37PM (#39586229)
Um, GraalVM (JIT) has both GC and memory safety. It's a drop-in replacement for HotSpot. It takes literally five seconds to change JAVA_HOME and you're done.

The benchmark literally uses Graal, something from Java, for Ruby and Python, but inexplicably doesn't use it for Java itself.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by kaba0 on Sunday March 03, 2024 @05:50AM (#39580023)
So you are comparing a feature-packed enterprise framework with a raw http server ping-ponging hello world?

Also, what even does the last line mean? Go in general is significantly more verbose than Java.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by parkcedar on Sunday March 03, 2024 @12:38AM (#39578716)
The fastest Java version is even beating his baseline of `cat` to `/dev/null`
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by JohnBooty on Sunday March 03, 2024 @02:04AM (#39579068)
Yes, though it's also worth noting that the fastest solutions are all doing their work in parallel which is not a thing for `cat`.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by timeagain on Sunday March 03, 2024 @01:36AM (#39578967)
More proof that the JVM is space-age future technology.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by neonsunset on Sunday March 03, 2024 @01:04AM (#39578827)
Still loses to .NET. On reference host Java still closer to 1.7-2s ballpark (and has to use awkward SWAR to get there) while the fastest solution in C# is 1.2s, beating C++ (code can be ported however).

But yes, "I expected Go to win..." is exactly the core of the problem here. Same as with e.g. Swift, which people expect to perform on the level of Rust, when it is even slower than Go. The intuition caused by common misconceptions just does not correspond to reality sadly.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by pjmlp on Sunday March 03, 2024 @03:35AM (#39579433)
It only goes to show how much cargo cult is there in adopting these languages in hipster circles.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by hnlmorg on Sunday March 03, 2024 @04:07AM (#39579550)
No. What it actually demonstrates is that people didn’t read the source material properly.

The Java and Go versions use different optimisations. There’s nothing stopping either language from using the same optimisations as the other. It just wasn’t something their respective authors cared to try in their respective exercises.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by neonsunset on Sunday March 03, 2024 @05:22AM (#39579899)
There, however, is something stopping Go from using optimizations present in Java or C#/C++/Rust. This is lack of SIMD API without dropping to hand writing ASM and overall much weaker compiler. This puts much greater burden on the programmer to match the performance while staying with Go.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by hnlmorg on Sunday March 03, 2024 @06:37AM (#39580213)
> There, however, is something stopping Go from using optimizations present in Java or C#/C++/Rust

...

> This puts much greater burden on the programmer to match the performance while staying with Go.

Your second statement contradicts your first. You're not stopped from using SIMD in Go. There are in fact several 3rd party libraries out there to use SIMD. It's just not part of the standard library. So you can still use SIMD in Go without writing Go's dialect of assembly.

It's also worth noting that SIMD isn't even due to drop into std in C++ until C++26. At the moment you either have to use experimental or a 3rd party library.

You're also missing the point of these examples. Nobody writing these exercises are trying to claim that all languages are equal. And they certainly not trying to write idiomatic code either. They're just fun little exercises demonstrating the degree of optimisations one can go through. You wouldn't want to write code like those in the examples in all but a the tiniest of scenarios.

It's silly the amount of people over-analysing this, what is essentially just a game, and then arguing its "proof" about their biases towards different programming languages.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by kaba0 on Sunday March 03, 2024 @04:05AM (#39579542)
> beating C++

Source for that?

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by neonsunset on Sunday March 03, 2024 @04:08AM (#39579553)
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by threeseed on Sunday March 03, 2024 @01:07AM (#39578839)
JVM has always been on par if not often faster than hand-written C code.

Go's advantage has always been that it is good enough at a lot of things.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by ffsm8 on Sunday March 03, 2024 @01:57AM (#39579047)
I think the reason why this misconception is so widespread is because there is a grain of truth in it, because almost everyone sees Java synonymous with gigantic framework like spring, quarkus etc.

In go you've got your standard libraries, these are generally quicker than the Java equivalent simply because they do less in the lifecycle of the operation.

This lets Java do funky stuff like enabling full jvm/code tracing just by adding a jar file at runtime. But it does come with a performance penalty.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by pjmlp on Sunday March 03, 2024 @03:38AM (#39579448)
Which is a reason why dynamic loading of agents now requires being enabled explicitly.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by _ph_ on Sunday March 03, 2024 @03:37AM (#39579443)
One has to differentiate here a bit. Java JIT technology has become really great. Highly optimized native code generation which hugely benefits from the ability to use live profiling data to optimize the code. This is why it often beats static compilers at generating faster code. The static compilers can only optimized on the range of possible data, the JIT can optimize based on the data presented to the program.

On the down side, there are quite a few features of the Java language and the JVM, which often make programs slow. Like a lot of details of the object model, lack of value classes, JIT compiling which takes time on startup etc. Also, a lot of Java libraries are pretty heavy weight.

Go is quite different here. It is statically compiled, which allows for fast program startup and the language model makes it quite easy to rather naively write programs which perform reasonally fast. The down side is, that the compiler is static and not so heavily optimizing as other static compilers for fast compilation speed. However recently the ability was added to use profiling data for optimizing the compilation.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by Mawr on Sunday March 03, 2024 @01:51AM (#39579029)
Please, you've been reading too much PR from the Java side and not looking at benchmarks and real-world performance enough. What you're claiming is inherently not possible, cherry-picked benchmarks notwithstanding.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by vanviegen on Sunday March 03, 2024 @02:04AM (#39579070)
Sure it's possible. The JVM can do guided optimizations at run time. There is no such thing for native executables.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by pjmlp on Sunday March 03, 2024 @03:43AM (#39579462)
And as I mentioned in another comment, you can even cache PGO data between executions, not needed to start always from zero.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by threeseed on Sunday March 03, 2024 @02:18AM (#39579134)
Can you explain why it's not technically possible.

JVM has had decades of experience at optimally translating bytecode to machine code and can take advantage of SIMD, AVX etc when needed. Most hand-written C code is far from optimal.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by 10000truths on Sunday March 03, 2024 @02:57AM (#39579281)
C compilers also have decades of experience optimally translating C code into machine code, and they are arguably more capable of emitting SIMD (good luck trying to use cutting edge AVX-512 intrinsics like vpopcntdq with the JVM). The fact is that there is nothing a JIT compiler can do that an AOT compiler can't do, but in the case of AOT, the resources spent compiling the code are amortized to effectively 0, whereas that resource cost is borne upon every program startup for a JIT engine.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by pjmlp on Sunday March 03, 2024 @03:42AM (#39579459)
C compilers only have one opportunity to do that once, at compile time, if the developer was lucky with their data set used to train the PGO output, maybe the outcome is greatly improved.

Modern JVMs, not only have the JIT being able to use actual production data, they are able to cache PGO data between execution runs, and reach an optimimal set of heuristics throughout execution time.

And on Android, those PGO files are even shared between devices via Play Store.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by neonsunset on Sunday March 03, 2024 @04:08AM (#39579555)
That's not necessarily true, on JIT vs AOT split. I'm mostly going off of how the divergence in available optimization is starting to look like in .NET after introduction of Native AOT with light research into LLVM and various optimization-adjacent Rust crates.

In particular, with JIT, you are able to initialize certain readonly data once, and then, on recompilation to a more optimized version, bake such data as JIT constants right into emitted machine code. This is not possible with AOT. Same applies for all kinds of in-runtime profiling/analysis and recompilation to incorporate a collected profile according to this exact run of an application. JIT also offers the ability to load modules dynamically in the form of bytecode without having to have a strict machine-level ABI, only the bytecode one, which allows for efficient generics that cross modules, as well as cross-module function inlining. And last but not least - there is no need to pick the least common denominator in supported hardware features as the code can be compiled to use the latest features provided by hardware like AVX512.

On the other hand, pure AOT means a frozen world which allows the compiler to know exact types and paths the code can take, performing exact devirtualization and much more aggressive preinitialization on code that accepts constant data. It also means bigger leeway in the time the compiler can spend on optimizing code. Historically, GCC and LLVM have been more advanced than their JIT counterparts because of different tradeoffs more favouring to absolute performance of the emitted code as well as simply higher amount of man hours invested in developing them (e.g. .NET punches above it's weight class despite being worked on by a smaller team vs OpenJDK or LLVM).

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by dzaima on Sunday March 03, 2024 @12:29PM (#39582375)
A couple weeks ago I managed to get a nice setup for viewing Java JIT disassembly[1], and did a bunch of ad-hoc tests on OpenJDK 21 (and some on whatever build of 23 was on openjdk.org). do manage to vectorize a decent amount of vectorizable loops, but semi-often missed some improvements, and some trivial things didn't vectorize at all (most amusingly, a loop summing an int[] to an int didn't get vectorized, while int[] to long is). Scalar code codegen is also quite mediocre.

GCC & clang, in comparison, often produce assembly which I'd consider optimal, or close to it, for vectorizable loops, and are noticably better than OpenJDK for scalar code.

[1]: it's a bit messy to use but: https://github.com/dzaima/grr#java-jit

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by saagarjha on Sunday March 03, 2024 @04:02PM (#39584516)
GCC and clang produce far from optimal code when vectorizing. Anyone doing serious work is unlikely to rely on the autovectorizer without consulting the output religiously.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by dzaima on Sunday March 03, 2024 @06:33PM (#39585518)
There are certainly a lot of things they may do suboptimally (in my experience: when there's register pressure; things that aren't nicely elementwise (bit arrays); preparation & tail handling can get pretty excessive; basically no tuning for making better use of ports) and of course you should always be checking the output (this is true for any optimization effort, vectorized or not), but for most things IME they do just fine (assuming the thing in question is vectorized at all). At the very least it's still miles above OpenJDK.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by saagarjha on Monday March 04, 2024 @06:00AM (#39589023)
I've only really seen decent results on the simplest of examples, like a bounded loop with a basic operation in it (like, adding all the numbers in an array). As soon as you go beyond that I see much worse code: reloading things inside the loop body that really should have been hoisted out, spilling for no good reason, weird contortions to shove certain things into scalar registers in the middle only to take them out again. And, of course, if your control flow is not trivial the compiler will often not recognize the loop as vectorizable at all.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by dzaima on Monday March 04, 2024 @07:02AM (#39589425)
Unknown-bound loops are not vectorizable in general (at least not sanely; you'd need masked stores, aligned writes, and a bunch of handling for different cases of bound computation). I haven't seen many cases of missed hoisting (outside of register pressure, where it's not really "missed") or pointless moving to scalar (at least with clang; I've looked at gcc much less), nor control flow be much of an issue either, both compilers can handle conditional loads & stores where available.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by samatman on Sunday March 03, 2024 @01:54PM (#39583267)
Of course it's inherently possible. The code is running on the same chip, it is inherently possible for a Foo compiler to emit the same machine code as a Bar compiler for the same algorithm. Foo being Java and Bar being C doesn't change this.

You might mean it's impractical? Or that it happens to not be true in the general case?

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by Shorel on Tuesday March 05, 2024 @07:24AM (#39602433)
> JVM has always been on par if not often faster than hand-written C code.

I have never found this to be true.

There are a few programs (Talend, MySQL workshop) written in Java that I sometimes have to use, and I avoid them as much as I can, because they are slow, bloated, and eat lots of memory.

Java new operator is faster than malloc, because Java allocates all memory needed from the OS before starting the program. And malloc is extremely slow. C++ new operator uses malloc so it is also slow.

Stack allocation in a C program is hundreds of times faster than using new in Java.

And that's for one feature, there are many other features like C++ compile-time calculations in templates that simply have no Java equivalent.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by tutfbhuf on Sunday March 03, 2024 @01:07PM (#39582823)
Your expectation is correct. The Java version is more tuned. Here https://github.com/dhartunian/1brcgo/?tab=readme-ov-file#lea..., you can find a version that runs in 1.108 and is almost 3x better than the one you quoted. I think one can reduce it even further. In the end, it depends on how fast the executable can boot up and execute the code. At some point, JVM will lose because it takes quite some time just to initialize the JVM, whereas Go executables can boot up very fast. Here you can see a comparison of Hello World Programs: https://github.com/qznc/hello-benchmark?tab=readme-ov-file#h.... JVM takes a whopping 596 ms to boot up and print Hello World, whereas Go just requires 149 ms.
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by benhoyt on Sunday March 03, 2024 @04:13PM (#39584600)
I think that's true with the JVM, but the fastest Java solutions are using GraalVM and its ahead-of-time compilation mode to avoid startup time. In addition, while "go run" might take 149ms to compile and run a program, a compiled Go program starts in just a couple of milliseconds:

  $ time ./t
  Hello, world
  
  real  0m0.003s
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by tutfbhuf on Monday March 04, 2024 @03:00AM (#39588062)
Thank you for the insights. I would like to add that native compilation with GraalVM sounds excellent in theory, but in all the real-world Java codebases that I have tested for native compilation, it wasn't possible due to some limitations in the code or some dependency (it is not a drop-in replacement).
Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by geodel on Sunday March 03, 2024 @02:28PM (#39583606)
Java version is written by lead researcher and founder of GraalVM at Oracle labs. It is really native AOT compiled code comparable to best in C++/Rust.

It is Java entry because language used is Java but finally compiled artifact is far far away from typical compiled Java artifact.

Re:The One Billion Row Challenge in Go: from 1m45s to 4s in ... by bufferoverflow on Sunday March 03, 2024 @08:47PM (#39586275)
3x difference is not "close".