With the API being nearly the same, I keep just thinking that Virtual Threads are basically identical to Platform Threads except that they use far less memory (so you can have lots more of them).

Are there any other actual differences? Better Peformance?

The context switch time is much smaller, so yes, better performance.

The relationship between throughput, latency, and concurrency in servers is expressed via Little's theorem. If your server is written in the thread-per-request style -- the only style for which the platform offers built-in language, VM, and tooling support -- then the most important factor affecting maximum throughput is the number of threads you can have (until, of course, the hardware is fully utilised). Being able to support many threads is the most effective improvement to server throughput you can offer.

See: Why User-Mode Threads Are Good for Performance https://youtu.be/07V08SB1l8c

Thanks for the video. I feel like there's a bit of conflation between the terms "performance(latency)" and "throughput", but I see the point. I'd be interested to see that latency graph (Time marker 15:38) between platform and virtual threads in the case where the server doesn't manufacture a 100ms delay (say, in the case of a caching reverse-proxy).

Also - millions of Java programmers thank you for not going to async/await. What an evil source-code virus (among other things that is).

I tried to watch it at 1.25x speed as I normally do, but you already talk at 1.25x speed, so no need !

To understand what happens when the server doesn't perform IO, apply Little's formula to the CPU only. Clearly, the maximum concurrency would be equal to the number of cores, which means that in that situation there would be no benefit to more threads than cores. What you would see in the graph would be that the server fails once L is equal to the number of cores. The average ratio between IO and CPU time as portions of the average duration would give you an upper limit on how much more throughput you gain by having more threads. That's what I explain at 11:34.

Also, both throughput and latency are performance metrics.

I watched the video and thoroughly enjoyed it, thank you for sharing it! I have a question that is perhaps not entirely related to the video, but it touches the topic of context switches. I've read this post [1] by Chris Hegarty, which explains that when calling the traditionally blocking network I/O APIs in the Java stdlib from a virtual thread, it uses asynchronous/poll-based kernel syscalls (IOCP, kqueue, epoll on Windows, Mac and Linux respectively) which I assume is to avoid blocking the carrier threads. That post was written in 2021, does it still hold true today in Java 21?

Reading that, it also makes me wonder what happens for disk I/O? Many other runtimes, both "green thread" ones like Golang and asynchronous like libuv/tokio, use a blocking thread pool (static or elastic) to offload these kernel syscalls to because, from what I've read, those syscalls are not easily made non-blocking like e.g epoll is. Does Java Virtual Threads do the same, or does disk I/O block the carrier threads? For curiosity, does Java file APIs use io_uring on Linux if it is available? It is a fairly recently added kernel API for achieving truly non-blocking I/O, including disk I/O. It doesn't seem to bring much over epoll in terms of performance, but has been a boon for disk I/O and in general can reduce context switches with the kernel by reducing the amount of syscalls needed.

[1]: https://inside.java/2021/05/10/networking-io-with-virtual-th...

> That post was written in 2021, does it still hold true today in Java 21?

Yes.

> Does Java Virtual Threads do the same, or does disk I/O block the carrier threads? For curiosity, does Java file APIs use io_uring on Linux if it is available?

We're working on using io_uring where available, especially for filesystem IO. For now, filesystem IO blocks OS threads but we temporarily compensate by increasing the size of the scheduler's worker thread pool.

In late 2021 I compared OS threads to io_uring for filesystem I/O at random-access reads from fast, NVMe SSDs.

That measurement told me that it's not necessary to use io_uring for disk I/O performance for some workloads.

It found no improvement in performance from io_uring, compared with a dynamic thread pool which tries to maintain enough I/O-blocked threads to keep the various kernel and device queues busy enough.

This was a little surprising, because the read-syscall overhead when using threads was measurable. preadv2() was surprisingly much slower than pread(), so I used the latter. I used CLONE_IO and very small stacks for the I/O threads (less than a page; about 1kiB IIRC), but the performance was pretty good using only pthreads without those thread optimisations. Probably I had a good thread pool and queue logic, as it surprised me that the result was much faster than "fio" banchmark results had led me to expect.

In principle, io_uring should be a little more robust to different scenarios with competing processes, compared with blocking I/O threads, because it has access to kernel scheduling in a way that userspace does not. I also expect io_uring to get a little faster with time, compared with the kernel I tested on.

However, on Linux, OS threads* have been the fastest way to do filesystem and block-device I/O for a long time. (* except for CLONE_IO not being set by default, but that flag is ignored in most configurations in current kernels),

> less than a page; about 1kiB IIRC

Interesting, didn't realize the kernel would let you do that. I guess it makes sense since it's up to user space to map pages for the stack. The kernel doesn't have much to do on clone except set the stack pointer.

That is an absolutely amazing video. From a brief, intuitive, and well-diagrammed explanation of non-trivial concepts of queuing theory, to practical examples, to connecting it all to real-world use cases and value, and all within a surprisingly short period of time, it is one of the most impressive technical videos I've ever seen. Thank you.

"With the currency being the same, I keep thinking a salary of $50k/year is the same as that of $500k/year. Are there any other actual differences?"

Just as with performance improvements [1][2][3][4], the actual impact on the user experience is non-linear and often hard to predict. In the case of virtual threads, you go from needing to consciously work around a limited amount of available threads to spawning one per request and moving on.

[1]: https://youtu.be/4XpnKHJAok8?t=3026

[2]: "These tests are fast enough that I can hit enter (my test-running keystroke) and have a response before I have time to think. It means that the flow of my thoughts never breaks." - https://news.ycombinator.com/item?id=7676948

[3]: https://news.ycombinator.com/item?id=37277885

[4]: "Go’s execution tracer has suffered from high overhead since its inception in 2014. Historically this has forced potential users to worry about up to 20% of CPU overhead when turning it on. Due to this, it's mostly been used in test environments or tricky situations rather than gaining adoption as a continuous profiling signal in production." - https://blog.felixge.de/waiting-for-go1-21-execution-tracing...

Structured concurrency in JDK 21 is not only a powerful and flexible library feature, but one that is built deep into the runtime in a way that allows observability into the relationships among threads: https://openjdk.org/jeps/453

Do you know if clojure will be adopting these virtual threads, or still using the macro-based approach?

You can adopt them yourself in Clojure by changing the executors Clojure uses. You can do this for both agents and core.async.

In my use case, when I tested this, core.async + virtual threads had some slight performance benefits over agents + virtual threads, but since core.async relies on macros the error messages can be hairy.

Thus I prefer to use agents. But my use case involves tens of thousands of these things.

It depends upon what you mean here.

You could always set the backing thread pools for core.async and agents in Clojure. That gives you the ability to use virtual threads right now.

But in order to avoid thread pinning, there will need to be some code changes to convert some uses of synchronized to ReentrantLock. How fast that happens will depend upon the given library maintainer. Here's an issue for making some of these changes in Clojure's core library: https://clojure.atlassian.net/browse/CLJ-2771

I've tested Clojure's agents with the new virtual threads for my targeted use case they're significantly faster than before - I can spin up tens of thousands of mostly idle agents and reach performance close enough to core.async for me.

It's somewhat unfortunate that structured concurrency ended up being a preview feature in 21. I agree that it's a great addition but man it'd be nice if it made the LTS.

As it stands, probably won't be heavily used until Java 25.

Organisations that care about new features need to reconsider their stance on using Preview features and even more importantly sticking to versions for which the sales org offers an LTS service. The whole concept of LTS is designed for companies that are so uninterested in new features (often because their software is no longer heavily maintained) that they're willing to pay money not to receive them. There are a lot of such projects around, and the fact that non-legacy projects choose old versions and an LTS service shows that the ecosystem still hasn't adapted to the new release model.

That seems like a bad faith interpretation. Upgrading has both costs and risks. Even upgrades within the same major version can break things. LTS is about paying for stability, not a lack of features.

Well, yes, but in the past the versions that now get a new integer number (feature releases) were mandatory for everyone and there was no LTS at all. There were some differences, but not as big as many think. The biggest one was the psychological aspect of the name (7u4 or 8u20, which were not patch releases but big feature releases).

So why did we create the LTS service? 1. Because the new feature releases, while no more risky than the old ones (like 7u4 and 8u20), do require a little more work that companies don't want to put into legacy applications, and 2. Many companies indeed are willing to pay for more stability for their legacy apps.

So while it is absolutely true that some projects want better stability, this level of stability is new. Companies that religiously stick to old versions now didn't do that in the past. The simplest explanation is that the new release model isn't yet understood, not that thousands of companies changed their risk strategy.

While I tend to agree, it's just a losing battle (in my experience). The issue is that shockingly few people in organizations actually care about language improvements. Further, because orgs prioritize spitting out new features above all else, selling maintenance work like updating the JVM is seen more as pure waste than actual benefit to the company.

The one argument I've been able to make to get an update is "This has fallen out of support and will no longer get security updates". That seems to be the only motivator for my company to do updates.

Congrats to you and the team on this huge milestone!

Really looking forward to taking advantage of these things (transparently and automatically!) in ZIO/Scala... which I think shows the true power of the JVM-as-platform approach you're taking!

Why haven't places updated already? It's not that much work to update. Where I work we always go to the new LTS version as soon as it's supported by gradle.

Doesn't cross anyone's mind to _not_ upgrade.

1) dependencies need to be upgraded. for example, not all versions of Gradle support all Java versions. So you need to upgrade Gradle to upgrade Java.

2) other things are deemed to have higher priority.

3) people are satisfied with existing features and don't want to spend energy to upgrade to something that doesn't provide immediate value.

4) folks aren't educated on what the benefit of switching would be so why would it be prioritized? This is a case of "they don't know what they don't know".

I work on a team using Java 8 daily. It's fine. It's got things I wish it didn't (no null in switch statements for example) but I don't care about that so much that I'm going to go through the pain of upgrading 7-9 services in the mono repo, their dependencies, and then test them all to be on a new version of Java.

Gradle/groovy is a liability for any jdk upgrades (similarly like lombok, but it usually supports new JDK at release, not before).

We ditched spock because of groovy, and never looked back. Now at jdk 21, previously at 20.

That's pretty disingenuous. Groovy has always run on newer JDK versions before they even get released.

One year ago, Gpars already supported Virtual Threads: https://groovy.apache.org/blog/gpars-meets-virtual-threads

As a heavy user of Groovy/Spock, though, I agree that upgrading Groovy itself can be challenging, unfortunately. Really depends though on how many edgy Groovy features you relied on :).

Not always for sure. We started JDK upgrades with 9 and went +1 every half year and Groovy was lacking with one of 10, 11, 12 or 13. It got so tiring that we had to let it go. Fortunately our tests were mostly JUnit 5, so it wasn't much of work.

We only used it for Spock AFAIR.

If you're upgrading every minor Java version, then yeah, I agree Groovy and most other dependencies that may not work on Java version changes off the bat (Lambok, probably Spring and other heavy frameworks like Micronaut and Quarkus, build tools like Gradle... many more) are going to slow you down. You end up with a very simple project if you remove all of that, which is actually a good thing if you can afford doing it.

Those are not minor versions, that's quite natural path and is supported by every lib we used, except groovy. And this is the encouraged path for JDK upgrades, people are lazy, but not us :)

Spring supports new JDK release ("minor version" like you called them, those between 11 and 17 and 21) before release. The only exception was with JDK 13, there was about 2 week slip there.

Lombok (I don't like it) supports every such version at release (not before unfortunately).

Other libs didn't even error out (we keep them at newest versions possible, aside from Jakarta madness).

So from the major libs/frameworks, the only thing that slowed us down was groovy.

I see your point... I've never upgraded to the non-LTS Java versions (almost no one is doing it - as you can see in any survey about it - which in itself is enough for me not to be the "brave" one doing it!) so I don't know how much pain that would've been. To me that just shows that Groovy could do with a bit more attention as I find it to be a great language, specially with Spock... it has made our tests so much nicer. It does have some warts but those are mostly a result of the lack of attention it receives, unfortunately, not some fundamental issues with the language.

It is a bit brave to upgrade every 6 releases or every 4, I prefer to have smaller issues more often than big ones rarely.

But I get that most of the time the issues is with dependencies that upgrade to lazily (that's why I don't like those that are bytecode magic and don't use ASM).

Kotlin, not Groovy, has been the culprit for slower Gradle support. I believe there was some split module pain in Groovy wrt Java 9, but it has been very smooth since then. The Kotlin compiler on the other hand is not very forgiving.

This is a moot point because your the build execution and the project compile/run can be on different JDKs. It is a tiny amount of configuration to decouple them, e.g. to use an EA build.

Yeah, you are right that decoupling build tool JDK and compile JDK is the way to go.

But Groovy does indeed not work, or has support very late for releases between 8, 11, 17 and 21 - so for anyone that wants to stay current (and not wait 3 or 2 years), using groovy in your code will be a pain - that might be also possible for other JVM languages, but I don't know, haven't used them.

Oh interesting. Typically the gradle release notes, like 8.4-rc-1, will have comments that imply that Groovy build scripts compile but Kotlin ones lag.

"Gradle now supports using Java 21 for compiling, testing, and starting other Java programs. This can be accomplished using toolchains. Currently, you cannot run Gradle on Java 21 because Kotlin lacks support for JDK 21. However, support for running Gradle with Java 21 is expected in future versions."

https://docs.gradle.org/8.4-rc-1/release-notes.html#support-...

1) is garbage. Since grade 6 you can run on 22 ea with no issues. Use toolchains, as they say on their docs.

2) no shit. What business user is every in their mind prioritising upgrading their language version? It's not up to them to push the upgrade. It's yours.

3) of course they are. People don't desire what they don't want. Invest in people who are actually interested in improvement of their software.

4)the java team have been pushing heavily via twitter / youtube / infoq / hacker news / other open jdk providers all the new features for every single java version during their 6 months release cycles. If your devs / your team don't know about it, then maybe again youre not encouraging people to want to improve on what they have, or take interest in the tech they work in.

I mean that is fine, do I give a shit what java version in using for my take home salary? No...but I enjoy using the newest, most interesting and useful tools. And you best believe those people are more attractive to other companies and you working on some 15 year old java 8 tech.

1) you're assuming every one is on Gradle 6 or higher lol. I can assure you that is not true.

2) Sure, pushing and making the decision are not the same thing. I can complain and persuade as much as possible and it doesn't mean it's going to happen.

3) I agree that you want people who care about improving software. Upgrading language versions isn't always the best route to do that though.

4) I don't think everyone on the team is reading about the latest updates in the world of Java. I think a pretty small portion of engineers are keeping 100% up to date, following Twitter accounts for Java dev, watching YouTube videos on it, etc. All that is educational and that's great to know but for most people, it's not going to help them work better because they know about features they can't use.

5) definitely sounds aggressive but okay. I haven't found a company yet who's complained about working in Java 8 versus 11/17. If a company is hiring for a role that uses Java, they're likely not limiting their candidates to those who've used their version of Java. It's a pretty standard language and if you know any other object oriented language, you'll be fine.

You’re just delaying and making the upgrade worse when the time comes. It’s much easier to upgrade now to 11 and then 17 and then 21 rather than try to upgrade from 8 to 27 when 8 is finally EOL.

Whether you perceive there to be no immediate benefit (hint: there is, Java 8 is an antiquated runtime) or not, delaying upgrading until Java 8 EOL is a way larger risk than upgrading now.

I've never done a language upgrade. I don't know what it takes. I've heard 8->11 is painful and 11->17 is not. So doesn't that mean jumping from 8->17 directly is mostly the same as going from 8->11?

I'm not saying there is no benefit. I'm not saying there is no risk. I agree that going from 8->17 would be worse than 8->11->17.

My point is to list out reasons why a team may not be able to just spend a day upgrading (dependency issues) or why someone might not be given the time to do it.

This is a niche case, but I spent months trying to upgrade one of our services from one LTS version to the next (I forget which). We encountered a weird bug where services running on the latest JRE would mysteriously corrupt fields when deserializing thrift messages, but only after running for a little while.

After an enormously unpleasant debugging cycle, we realized that the JIT compiler was incorrectly eliminating a call to System::arrayCopy, which meant that some fields were left uninitialized. But only when JIT compiled, non-optimized code ran fine.

This left us with three possible upgrade paths:

* Upgrade thrift to a newer version and hope that JIT compilation works well on it. But this is a nightmare since A) thrift is no longer supported, and B) new versions of thrift are not backwards compatible so you have to bump a lot of dependent libraries and update code for a bunch of API changes (in a LARGE number of services in our monorepo...). With no guarantee that the new version would fix the problem.

* File a bug report and wait for a minor version fix to address the issue.

* Skip this LTS release and hope the JIT bug is fixed in the next one.

* Disable JIT compilation for the offending functions and hope the performance hit is negligible.

I ultimately left the company before the fix was made, but I think we were leaning towards the last option (hopefully filing a bug report, too...).

There's no way this is the normal reason companies don't bump JRE versions as soon as they come out, but it's happened at least once. :-)

In general there's probably some decent (if misguided) bias towards "things are working fine on the current version, why risk some unexpected issues if we upgrade?"

did you work for a very large rideshare company by any chance?

I encountered a weird bug with deserializing JSON in a JRuby app during an OpenJDK upgrade - it would sporadically throw a parse error for no apparent reason. I was upgrading to OpenJDK 15, but another user experienced the same regression with an LTS upgrade from 8 to 11.

The end result of my own investigation led to this quite satisfying thread on hotspot-compiler-dev, in which an engineer starts with my minimal reproduction of the problem and posts a workaround within 24 hours: https://mail.openjdk.org/pipermail/hotspot-compiler-dev/2021...

There's also a tip there: try a fastdebug build and see if you can convert it into an assertion failure you can look up.

fastdebug is a good tip, thanks for sharing!

The services I work on pump the entire business revenue from start to finish. A few nice to haves for devs aren't any where close in the risk calculation if something breaks

Sounds like code that's worth learning how to test.

I know right. Unfortunately with jdk upgrades only e2e tests have much value, and those are much more expensive

The bigger the project, the more painful the upgrade. Package systems are convenient to avoid reinventing the wheel, until you have to upgrade any piece of it. Then you're stuck trying to figure out which versions of each package go together.

If Package A won't run on JDK 17 your entire project is stuck on JDK 11. If Package B is upgraded but has conflicts with Package A, you have to dig through old versions until you find one that works -- and you don't get upgrades.

The more games somebody has played with reflection, undocumented features, deprecations, etc. the more likely you are to have a conflict. And since package managers encourage you to depend on somebody else's code, you end up depending on everybody else's code.

The smaller and greener the project is the more likely it is you can just pull the latest versions and be happy about it. A project that was written when Java 8 was current, and continued to develop, is going to be a nightmare.

"Oh look, I need to upgrade mockito and Spring. Oh, now I upgraded Spring I need to update the spring JPA plugin. Oh now I upgraded that I need to upgrade Hibernate. Oh now I need to upgrade the library built on it that that team over there maintains. Oh, they're not interested." etc. etc.

Having a lot of experience with that stack... it is why i migrated to scala long ago

Tho Scala makes the upgrade problem even worse:)

When using Spring Boot you usually update just one version and everything else is updated via BOM. There should be a really good reason to have fine-grained control over every single dependency.

Not every app using Spring is using Spring boot

You don’t have to use its functionality to use its BOM.

So honestly I didn't realise that it's POM extended to ecosystem libraries beyond Spring's own. However, it still doesn't solve the problem that e.g. the hibernate version compatible with Spring Framework n+1 is not compatible with the hibernate version compatible with Spring Framework n and now you're doing an "all or nothing" upgrade, which for a large app can be time consuming.

The point of using BOM is to avoid specifying dependency versions of individual components, so this problem is in reality non-existent. The only case when you would actually need to have different version of Hibernate than in your BOM is some critical bug fix, which is very likely a patch version and is very unlikely to break compatibility with the rest of your setup.

True, Spring upgrades can be a pain in the ass. There is a trick to make it less painful though. Use maven BOM for version management. As with any framework upgrade, it doesn't make the process entirely painless. But very much less painful.

Java isn't forward compatible?

Overwhelmingly it is, until it isn't. There are tiny gotchas, especially if you play with some of the murkier aspects, such as reflection or class loading.

The more of someone else's code you use, the more likely one of them bumps into one of the gotchas. And that sets off a cascade of conflicting versions.

Other workplaces have more long term supported systems, with more complex requirements, and more SLAs and guarantees to fulfil.

For an example, my team owns a dozen services and they have hundreds of direct and transient dependencies. Of those, maybe a dozen or two to need work to support the new version but that's a dozen different teams that have to put the work on their roadmap and prioritize it. When the entitlement is 'devs want to use shiny feature X with hard to quantify productivity benefit' it's difficult to prioritize. When there's an efficiency benefit then things move fast because a 10% efficiency improvement means 10% lower server costs and that's easy math.

On topic of upgrades, here are a couple of tricks I've learned throughout the years which make the upgrade process easier.

1. Use Maven 2. Use BOMs to manage related dependencies 3. No lombok

You forgot about 8->17 which ads really nice language features, records alone are greatest feature after lambdas.

And 21 brings patterns in switch and records.

Do you have to baseline on Java 21 if you want to add support for virtual threads? Couldn't you continue using heavyweight threads on older versions of Java? My understanding is that both use the same Thread abstraction.

From an API perpective, you can always use reflection to cheat past the option to create virtual threads in pre-21 (without previews) java bytecode, but you need to do more to your code than just flip the switch to support virtual threads.

A virtual thread thread pool by definition is unbound. If you're binding data to a thread (eg. Thread locals, you now have a seemingly unbound list of threads that is now effectively a memory leak). I bumped into that one a few months ago with Netty that has a per thread cache for some things (thankfully you can turn off that cache). It was creating a significantly large waste of RAM that slowed down the application alone.

The other big one is as I mentioned the synchronized limitation. If you assume naively that anything can run in a virtual thread without worries, you're opening yourself up to deadlocks or at least significantly low performance code if you're relying on libraries/code that are synchronized using java monitors.

There may be more examples of gotchas, but these two are the most notable examples I have right now.

I believe, e.g. ZIO 2.next is doing something like this, dynamically deciding whether running something async or just doing the blocking thing depending on the availability of VThreads... but of course that's Scala, so YMMV.

Without a way to trampoline computation (or transform code appropriately) it's probably impractical to do anything like that.

(And of course, still many caveats as the sibling post points out.)

> Expect a lot of libraries to start release versions that are java 21 baseline because of this feature alone.

Java has had multi-version jars since 11 I think... that allows library authors to ship code that benefits from new features in newer versions of the JDK while still supporting older ones as well. Hopefully library authors can leverage that, though I'm aware something like Virtual Threads may be very difficult to design around for older versions.

mrjars are a terrible idea, because libraries are very rarely a single jar, and require dependencies. Often that dependency set is different depending on the runtime version because of backports of newer Java APIs. So either you don't care and make your consumers use proguard to remove the unnecessary backports, or you create a Maven package with variants for each runtime version, which 99% of your downstream will end up using anyway.