News for hackers. Stuff that matters.
Re:Disable transparent hugepages by markjdb on Tuesday November 28, 2017 @09:09AM (#15797385)
Please be aware that the article describes a problem with a specific implementation of THP. Other operating systems implement it differently and don't suffer from the same caveats (though any implementation will of course have its own disadvantages, since THP support requires making various tradeoffs and policy decisions). FreeBSD's implementation (based on [1]) is more conservative and works by opportunistically reserving physically contiguous ranges of memory in a way that allows THP promotion if the application (or kernel) actually makes use of all the pages backed by the large mapping. It's tied in to the page allocator in a way that avoids the "leaks" described in the article, and doesn't make use of expensive scans. Moreover, the reservation system enables other optimizations in the memory management subsystem.

[1] https://www.cs.rice.edu/~druschel/publications/superpages.pd...

Re:Disable transparent hugepages by drostie on Tuesday November 28, 2017 @09:56AM (#15797740)
Hey, thanks for being a FreeBSD dev. Every time I've been on a FreeBSD system my reaction has been "this is kinda weird, but really nice." (Especially, compiling from ports.) The fact that y'all are connected with academic communities who can solve these problems in more principled ways is really wonderful, as are the BSD/Solaris attitudes of "hey let's wall off things that don't need to interact" and "the kernel is the most important part, but it's not the only thing."
Re:Disable transparent hugepages by loeg on Tuesday November 28, 2017 @12:59PM (#15799644)
It's worth pointing out that the FreeBSD implementation (on AMD64) only promotes 4kB pages to 2MB pages and doesn't transparently promote to 1GB pages.

Given alc@ was an author on the paper (and the paper's FreeBSD 4.x implementation supported multiple superpage sizes), I'm not really sure why FreeBSD's pmap doesn't have support for 1GB page promotions.

Re:Disable transparent hugepages by kev009 on Tuesday November 28, 2017 @03:01PM (#15800863)
F5/LineRate did it but it got NACKed in a fairly underhanded and unfortunate way on the mailing lists :/ https://github.com/Seb-LineRate/freebsd/commits/seb/stable-1...
Re:Disable transparent hugepages by markjdb on Tuesday November 28, 2017 @03:32PM (#15801206)
That patch set does not implement transparent creation of 1GB mappings. It also contains dubious things like this, which make me think the branch was a WIP: https://github.com/Seb-LineRate/freebsd/commit/66a8d3474d410...

The only mailing list thread I see regarding this is here, and it doesn't seem particularly underhanded to me: https://lists.freebsd.org/pipermail/freebsd-hackers/2014-Nov...

Re:Disable transparent hugepages by kev009 on Tuesday November 28, 2017 @07:00PM (#15803246)
All the technical critique seems fair but it seemed like they (both as an individual and as a company) were a first time contributor and no outreach was really done to pull them in further. I guess LineRate imploded within F5 so there could have been structural problems inside there prevented them from doing a fully baked contribution anyway.
Re:Disable transparent hugepages by loeg on Tuesday November 28, 2017 @03:36PM (#15801238)
Can you clarify what you mean by "underhanded and unfortunate way?" The thread I can see on freebsd-hackers@ had a little feedback, including some very valid critiques (the code was developed against 9.x, then ported to 10.x without testing, at a time when 11 was CURRENT) and the original author just didn't follow up at all:

https://lists.freebsd.org/pipermail/freebsd-hackers/2014-Nov...

But maybe I'm missing something?

See also: https://lists.freebsd.org/pipermail/freebsd-hackers/2013-Sep...