Re:Java Virtual Threads Ate My Memory: A Web Crawler's Tale ...
by 1718627440 on Tuesday June 03, 2025 @06:24AM
(#44168372)
> You only have one shared heap
Can't you use mmap then?
> In a real-world program, you may be doing other things besides downloading URLs and feeding the URLs to the download scheduler. Even though the URL downloader is what's causing the memory pressure, an OutOfMemoryError may very well be thrown by an allocation anywhere else.
But upon that OOM, can't you still release the memory from the downloader so that you have memory for your other allocation?
Can't you use mmap then?
> In a real-world program, you may be doing other things besides downloading URLs and feeding the URLs to the download scheduler. Even though the URL downloader is what's causing the memory pressure, an OutOfMemoryError may very well be thrown by an allocation anywhere else.
But upon that OOM, can't you still release the memory from the downloader so that you have memory for your other allocation?