Java Virtual Threads Ate My Memory: A Web Crawler's Tale of Speed vs. Memory

Re:Java Virtual Threads Ate My Memory: A Web Crawler's Tale ... by jandrewrogers on Saturday May 31, 2025 @01:17AM (#44142005)

Proper async architecture requires designing a scheduler appropriate for your workload. Many (most?) async applications ignore or skip this part and try to delegate it to a runtime or the OS, usually with mediocre results. Designing a robust scheduler usually requires excellent visibility into the real-time distribution of available resources so that you can dynamically schedule load away from the resources under the most pressure at every instant in time. If you aren’t going to do that work, async may not be the best architecture choice.

[ Parent | on HN ]

Re:Java Virtual Threads Ate My Memory: A Web Crawler's Tale ... by pron on Saturday May 31, 2025 @04:59PM (#44146886)

First, this is true in any architecture. Second, in Java you can do just that with the use of semaphores guarding different resources with a different number of leases.

The one resource you can't easily schedule well is CPU, but that's not a big problem because interactive servers don't work well when the CPU needs to be scheduled (i.e. when it's at 100%). Again, this is true regardless of architecture.

Back when we first released virtual threads, many asked us why we didn't make the default virtual thread scheduler offer time-sharing (with or without priorities). The answer was that we wanted to, but were unable to find workloads for the intended domain of the default scheduler (interactive servers) where any kind of CPU scheduling could help (we said that when somebody finds such real workloads, we'll add fairer CPU scheduling to the default scheduler, assuming there was a good algorithm that would address such workloads).

[ Parent | on HN ]

Re:Java Virtual Threads Ate My Memory: A Web Crawler's Tale ... by jandrewrogers on Saturday May 31, 2025 @08:47PM (#44147905)

This is kind of what I am talking about though. The whole point of async scheduling is that semaphores, locking, etc are unnecessary because you strictly control the CPU schedule and can dynamically rewrite the schedule such that the need for such mechanisms largely doesn't exist. This provides a qualitative performance boost for quite a few analytics and other workloads, which is the only reason we do it (since it is considerably more complex to implement).

If you aren't bothering to write a real scheduler then it becomes less clear why you would bother with true async. Lots of hassle for not much payoff.

[ Parent | on HN ]