aybassiouny

I used to think this is where I am heading, until reading “Hunt, Gather, Parent”, and realizing that being a good parent does not mean my existence as an individual needs to stop. Me and my kids can co-exist. I just need to let go of controlling their lives.

It is an internal cache implementation. The library writer _thought_ this codepath would never be triggered - or at least very rarely, but that was proven wrong on the new hardware.

In my uninformed opinion: 1. waiting for 1 second sounds like a hack 2. The application should be doing cache writes asynchronously as as not to affect overall server latencies

I recommend to do more debugging next time and see which process exactly is costing you all this. It sounds like a classic memory leak - those can be triggered nondeterministically. Maybe try some of the tools suggested here https://mahdytech.com/2019/01/05/task-manager-memory-info/

The surprising part for me is, OS does not recover. As if once it tanks, it becomes more cautious about which memory is paged and which is not, even if there's plenty of free physical memory later.

Lucy, even in a healthy state, already have many page faults. They are just not on the hot path, e.g. logging. They don't affect latency much and nobody cares for them.

Turning off swapping has been a heated discussion, and I am on the negative side of it. Swapping is good; not all memory is important and some of it should not occupy precious physical space. Not all processes are important though. What I would rather have is a per-process swapping flag, allowing me to disable swapping for one or two essential processes, but swap the rest.

That is already tracked, but it is not very meaningful. We already have a lot of page faults (and paged memory), they are just not on the hot path.

Totally :) If I had a dime for every one who mixed working set and private bytes because of using Task Manager

Thanks for the mirror! It should be back up now.