News for hackers. Stuff that matters.
Re:I've had bad luck with transparent hugepages on my Linux ... by LinuxBender on Wednesday February 01, 2023 @08:25AM (#34610045)
THP can boost performance on apps that actually allocate massive amounts of memory under specific conditions but it's also had some issues with memory leaks specific to THP that are hard to spot. Have these been fixed? I've not seen any mention of it. I have always disabled THP on everything, especially machines with high memory pressure. On systems that I entirely engineer and control I over-provision memory so it's less of an issue, but on systems that I have provisioned for others I could not prevent them from using all the memory and that is where THP would get people into trouble. I've tried to encourage people to use cgroups with memory limits to no avail. Our work-around was to set fairly high min_free based on a formula suggested by Redhat and set overcommit ratio to 0 to create a safety buffer zone. I should add this was during the 3.x and 4.x kernel versions. I have not done any performance testing with THP on 5.x and rather just disabled it out of habit.
Re:I've had bad luck with transparent hugepages on my Linux ... by jeffbee on Wednesday February 01, 2023 @09:51AM (#34611204)
They’re not leaks, it’s just that the slop at the end of a present page is potentially 512 times more than it would be with 4K pages.
Re:I've had bad luck with transparent hugepages on my Linux ... by ckennelly on Wednesday February 01, 2023 @10:04PM (#34621517)
The default setting of max_ptes_none is also problematic.

On a stock kernel, it's 511. TCMalloc's docs recommend using max_ptes_none set to 0 for this reason: https://github.com/google/tcmalloc/blob/master/docs/tuning.m...

(Disclosure: I work on TCMalloc and authored the above doc.)