News for hackers. Stuff that matters.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by MaxBarraclough on Wednesday August 05, 2020 @08:00AM (#24059847)
Doesn't seem helpful to use the term green threads here. This isn't a JVM with green threads (those are a thing of the past as I understand it). They're using plain old OpenJDK, and they're ensuring the GC gets a CPU core to itself.

Neat that they were able to get a dramatic improvement in GC latencies on both G1 and ZGC.

No mention of the Shenandoah GC. Would the same trick help out there too?

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Wednesday August 05, 2020 @08:13AM (#24059933)
We did measure on Shenandoah as well, it helped but not enough to be within 10 ms. Since this post is about Hazelcast Jet getting the best latency, we didn't report that.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by MaxBarraclough on Wednesday August 05, 2020 @08:23AM (#24060010)
Interesting, thanks.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by brabel on Wednesday August 05, 2020 @08:08AM (#24059912)
> This isn't a JVM with green threads

Exactly... I thought they may be talking about Project Loom's Virtual Threads (which are going to be true green Threads) which are available experimentally as of Java 15, given they did use Java 15, but nothing in the post indicates they used that.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Wednesday August 05, 2020 @08:14AM (#24059942)
We use the same technique of cooperative multithreading under any name, but without the low-level support to be able to write plain sequential Java code. However, even though that changes our internal programming model, the behavior with respect to native threads, interactions with the OS scheduler, CPU caches, etc., should be identical.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Wednesday August 05, 2020 @09:37AM (#24060590)
Loom's virtual threads aren't cooperative, and the implementation actually does interact with the GCs in non-trivial ways. ZGC support was only added to Loom last week, and might still be unstable, but it would be interesting to test. Please report any finding -- be it performance or stability issues -- to the Loom mailing list.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Wednesday August 05, 2020 @10:08AM (#24060888)
"Virtual threads" are fibers? When you say they aren't cooperative, you mean there's JVM infrastructure that actually pre-empts a fiber?
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Wednesday August 05, 2020 @10:18AM (#24061026)
> "Virtual threads" are fibers?

Yes, but we found that the name "virtual thread" works better. For one, these fibres are actually an implementation of java.lang.Thread, so it's the same abstraction; for another, when people hear "fibres" they sometimes compare it to other implementations of fibres that are implemented very differently, and that causes confusion.

> When you say they aren't cooperative, you mean there's JVM infrastructure that actually pre-empts a fiber?

First, I mean that they don't require any explicit yields. Sometimes people confuse "preemptive" with "time-sliced preemption", so I'd rather just say "non-cooperative".

Second, there is a capability to forcefully preempt a virtual thread even if it's in some computation loop, but that's up to a chosen scheduler, and the scheduler isn't part of the JVM. You can supply your own scheduler, written in Java, and you'll be able to choose to preempt a runaway thread. Currently, this capability is not exposed, but it will be eventually.

See http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.h...

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Wednesday August 05, 2020 @10:35AM (#24061226)
> I mean that they don't require any explicit yields.

So in this sense, are Kotlin coroutines also non-cooperative? They don't explicitly yield, but do have a mandatory API to use for "blocking" calls.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Wednesday August 05, 2020 @11:17AM (#24061745)
They do explicitly yield, although not with an explicit keyword or call, but with an explicit type. I guess you could say that the "await" is inferred by the compiler, but it's there. That's not the case with virtual threads, which behave more like Go's goroutines or Erlang's processes.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Wednesday August 05, 2020 @02:09PM (#24063750)
>They do explicitly yield

Arguably, virtual threads also explicitly yield by calling one of the blocking methods in the JDK. This is very similar to putting all the bottom-level suspendable functions into the Kotlin standard library.

>virtual threads, which behave more like Go's goroutines or Erlang's processes.

I think this can be summarized as "Kotlin uses colored functions and Loom uses non-colored ones". This is a well-established core difference, I thought you had something else in mind with "explicit yield".

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Wednesday August 05, 2020 @02:46PM (#24064125)
> Arguably, virtual threads also explicitly yield by calling one of the blocking methods in the JDK.

There's nothing explicit here, or nothing more explicit than the ordinary platform threads you use today -- JDK operations might or might not block an OS thread. There is no way you can tell whether a call does that or not, and that behaviour can change (plus, there's forced preemption as an option).

> This is a well-established core difference, I thought you had something else in mind with "explicit yield".

I would say that an explicit colour for yielding qualifies as an explicit yield, and so falls under cooperative. In any event, virtual threads have neither explicit yield-sites nor special yield-site colours, hence they're non-cooperative, and they support forced preemption.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Thursday August 06, 2020 @01:48AM (#24068430)
Without the very last point, forced preemption, they would indeed be cooperative because not calling any blocking method would make them non-cooperative.

This is exactly the same as within the "colored" subspace in a language that has this distinction. As long as all the functions you call are suspendable, you equally have no idea which one will actually suspend.

So, without forced preemption based on GC safepoints, and as long as there is any blocking operation left in the library, Loom qualifies as a cooperative multithreading system.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Thursday August 06, 2020 @03:37AM (#24068876)
> Without the very last point, forced preemption, they would indeed be cooperative because not calling any blocking method would make them non-cooperative

If user code cannot possibly know whether an operation blocks or not, then it cannot "cooperate."

> As long as all the functions you call are suspendable, you equally have no idea which one will actually suspend.

First, this is true hypothetically but never in practice. Consider that if code really didn't care about blocking, then why not colour everything in the blocking colour? The answer is that in the coloured mode, compilation and cost of the two colours are very different.

Second, and more importantly, the reason that there are two colours is exactly to enable a cooperative style. While a "blocking" routine may or may not block, a non-blocking one never does and that is the crucial difference. With cooperative multi-tasking the default mode is that of a critical section -- there is no scheduling point unless you explicitly know in advance there might be one and where. With preemptive concurrency the default is the opposite: yielding may happen at any time unless you explicitly enter a critical section. This results in very different coding styles.

Anyway, we're arguing over definitions, so you may want to consult Wikipedia's definitions [1] [2].

[1]: https://en.wikipedia.org/wiki/Cooperative_multitasking

[2]: https://en.wikipedia.org/wiki/Preemption_(computing)#PREEMPT...

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Thursday August 06, 2020 @07:05AM (#24069861)
"processes voluntarily yield control periodically or when idle or logically blocked in order to enable multiple applications to be run concurrently."

This tells me virtual threads (without forced preemption) are cooperative.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Thursday August 06, 2020 @08:04AM (#24070177)
Virtual threads do not do it voluntarily. They have no knowledge or control over where they might yield. Without forced preemption, I guess you can say that as long as they don't call into the JDK in any way (including e.g. throwing exceptions) or any third-party library then they shouldn't normally expect to yield, but I don't count calling any code you haven't personally written "voluntarily yielding".

We call them preemptive with or without forced preemption -- in line, I think, with the definitions on Wikipedia -- but whatever you choose call them, the concurrency programming style is the same as that with threads today or Go's goroutines, and is different from the style of C#/JS's async/await, Kotlin's coroutines, or more explicit async code, all of which result in user code relying on knowing where yield points (possibly) are (i.e. "critical section" by default). BTW, even with OS threads, when you run transaction-handling code, as opposed to long-running computation, time-sharing preemption is the exception rather than the rule.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Thursday August 06, 2020 @08:55AM (#24070447)
I think the distinction is pretty clear: either the mechanism requires cooperation by the application thread (which typically initiates the yield at a compiled-in, predefined point), or it doesn't and the runtime environment preempts it from the outside.

Virtual threads are of the former kind. (At least as long as we don't involve the forced preemption feature).

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Thursday August 06, 2020 @07:33PM (#24076525)
Virtual threads are not cooperative, and OS threads that process transactions are also normally preempted almost exclusively at syscalls initiated by the thread, but I can't stop you from calling them that. The important thing to remember is that you program them like OS threads or Go goroutines or Erlang processes ("interleaving can happen anywhere unless I forbid it") and not like async/await or Kotlin's coroutines or asynchronous code or Windows 3.0 ("interleaving can only happen at certain allowed, known points"), whatever you want to call these two styles.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Friday August 07, 2020 @04:42AM (#24079751)
I concur with your point about the programming model and style, but I do also maintain that "cooperative" vs. "preemptive" is not about that difference. It is a technical difference on how the system interleaves threads and whether it needs cooperation from the code running on them, and not on the programming model and critical sections.

For the distinction you have in mind, I see the terms "colored" vs. "non-colored functions" to be used the most, and they are both within the "cooperative multithreading" space.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by nahuel0x on Wednesday August 05, 2020 @01:22PM (#24063219)
Can a Loom scheduler kill a virtual thread in an arbitrary execution point (like Erlang process but unlike goroutines)?
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by haxen on Wednesday August 05, 2020 @02:11PM (#24063773)
http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part2.h...

Yes it can and actually it's quite low-hanging fruit on the JVM. GC safepoints are already there, you just have hook into the mechanism.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Wednesday August 05, 2020 @02:48PM (#24064157)
True, but it heavily depends on what you mean by "can." Doing it safely in Java is a problem, as Java code does not protect from shared state. So I would say it's "sort-of, but not really, as it would be very dangerous unless you know exactly what the thread is doing." On the other hand, if the question is, if Erlang were implemented on the Java platform using virtual threads as processes, would code be able to kill a process arbitrarily, then the answer would be yes.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by nahuel0x on Wednesday August 05, 2020 @06:24PM (#24066266)
It can be useful for killing a clojure virtual thread who only uses shared memory by reading thread-safe persistent data structures and writes only to clojure atoms/STM (besides his unshared local state). If this is possible, then Loom + clojure can be a better model than erlang for some usages. Myriads of linked actors but with the added feature of shared memory for global views (see Rich Hickey criticism of the actor model) and optimized message passing (you don't need to copy messages if you have a global GC and they are clj persistent data structures). But external killing of a linked actor/vthread -one of erlang usually ignored secret sauces- is fundamental, if not, you need adhoc mechanisms like Go cancellation contexts who IMHO adds a lot of error prone accidental complexity. Think usages beyond supervisors/fault-tolerance like killing obsolete requests/computation or speculative execution.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Wednesday August 05, 2020 @08:06PM (#24066938)
The problem here is that Clojure only appears to other Clojure code to do what you're describing, but heavily relies on mutation and locking under the covers. Any lazy seq in Clojure is actually a mutable data structure that guards mutation with locks. Clojure, however, could emit instructions that check for interruption at sites that are safe for Clojure to interrupt a thread.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by nahuel0x on Wednesday August 05, 2020 @08:32PM (#24067079)
Just to understand you better, Clojure lazy-seqs are thread-safe but the Loom killing mechanism is not compatible with sections guarded with locks? So, if you had:

try { lock.lock() // long computation here, no interrupts check } finally { lock.unlock() }

What happens when the virtual thread is externally killed in the middle of the long computation? Nothing at all because is not manually checking for the interrupt token? (like Go, unlike Erlang). Or is interrupted but the finally block is not executed and we get a dangling lock? I know Loom is not finished yet, but I would like to know about his prospective.

Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by pron on Thursday August 06, 2020 @03:50AM (#24068940)
The forced preemption mechanism that uses VM handshakes doesn't care about locks, so it could hypothetically preempt and kill the thread inside the long computation. If you want to insert explicit interruption checks, that's another matter, and it doesn't require the forced preemption mechanism at all.
Re:Sub-10 ms Latency in Java: Concurrent GC with Green Threads by tyingq on Wednesday August 05, 2020 @08:13AM (#24059939)
They say they are using this: https://hazelcast.com/blog/idle-green-threads-in-jet/

And then say it's comparable. "This basic design is also present in the concepts of green threads and coroutines. In Hazelcast Jet we call them tasklets."