Re:Dealing with Out-of-Memory Conditions in Rust
by simcop2387 on Saturday August 14, 2021 @09:40AM
(#28180523)
Build perl with -DUSEMYMALLOC and -DPERL_EMERGENCY_SBRK, then you can preallocate a buffer by doing $^M="0" x 65536; then you can trap the out of memory condition with the normal facilities in language and handle it appropriately (mostly letting the big data get deallocated, or exiting). Then you can continue on just like normal. It's a weird setup and I don't think I've run into any other language with that built in.
[ Parent
| on HN ]
Re:Dealing with Out-of-Memory Conditions in Rust
by zbentley on Saturday August 14, 2021 @10:36AM
(#28180984)
Useful, but on Linux it's highly likely that by the time you're comparing $@ to ENOMEM the OOM killer has already awoken and is heading your way.
Re:Dealing with Out-of-Memory Conditions in Rust
by Xylakant on Saturday August 14, 2021 @11:12AM
(#28181255)
The interesting thing is that the OOM killer doesn’t always go for the program that triggered the OOM. It may also decide to kill another memory-hungry process (cough database) on the machine unless you explicitly tweaked it.
Re:Dealing with Out-of-Memory Conditions in Rust
by klyrs on Saturday August 14, 2021 @01:10PM
(#28182253)
If a program allocates all available memory, and systemd then hits OOM on a 1kB allocation, do you think we should kill systemd?
Re:Dealing with Out-of-Memory Conditions in Rust
by Xylakant on Saturday August 14, 2021 @03:08PM
(#28183160)
I’ve had t he OOM killer kill my work queue because someone wrote a shell script that leaked small amounts of memory. The entire purpose of that machine was to run the queue, so most of the memory was allocated to that process - making it the prime target for the OOM killer. Yes, please, kill everything else on that machine, before touching that process. (Indeed, the OOM killer has a setting for that)
Re:Dealing with Out-of-Memory Conditions in Rust
by klyrs on Saturday August 14, 2021 @03:30PM
(#28183333)
Sounds bad, but it doesn't justify the proposed heuristic to kill every new process that fails to alloc without looking at the whale that's responsible for the OOM condition. Actually, it sounds like you didn't properly configure that machine.
You might be happy to learn that the OOM killer can be configured[1] to specifically protect certain processes. If the entire point of a machine is to run a single process, then you should definitely use that feature.
Re:Dealing with Out-of-Memory Conditions in Rust
by zbentley on Saturday August 14, 2021 @01:32PM
(#28182438)
I mean, the OOM killer's heuristics are byzantine to be sure. However, if your program is not likely to be the "true" culprit of memory exhaustion, there are better tools at your disposal than ballast pages--cgroups and tunable oomkillers like earlyoom (https://github.com/rfjakob/earlyoom).
On the other hand, if you are likely to be identified as the culprit, I think the best you can hope for is getting some cleanup/reporting in before you're kill-9'd.
Re:Dealing with Out-of-Memory Conditions in Rust
by jamesmishra on Saturday August 14, 2021 @06:41PM
(#28184567)
I think you've identified the primary exception to the parent comment's rule. Obviously we would not want to kill PID 1, no matter the reason for the OOM.
Re:Dealing with Out-of-Memory Conditions in Rust
by jbverschoor on Saturday August 14, 2021 @10:42AM
(#28181023)
The next thing OSes will do is use memory compression to go against this behavior.
Behold, CS is as bad as governments with all their tax rules