Re:What Postgres Full Text Search Is Missing
by nickpsecurity on Tuesday August 06, 2024 @09:25PM
(#41177336)
Can this be fixed with alternative GC’s or tuning?
Re:What Postgres Full Text Search Is Missing
by fizx on Wednesday August 07, 2024 @12:05AM
(#41178145)
It might be better by now with the newer GC options (ZGC, G1, Azul). For a while those had their own problems, but I'm a little out of the loop.
Tuning the older options wasn't really all that beneficial. We tried tuning, then running a custom build of OpenJDK to mess with the survivor space (which isn't that tunable via config), then ultimately settled on more aggressive (i.e. weekly) rolling restarts of servers.
To make generational GC efficient, you want to have very short lived objects, or objects that live forever. Lots of moderately long-lived objects is the worst case scenario, as it causes permanent fragmentation of the old GC generation.
Lucene generally churns through a lot of strings while indexing, but it also builds a lot of caches that live on-heap for a few minutes. Because the minor GCs come fast and furious due to indexing, that means you have caches that last just long enough to get evicted into the old generation, only to become useless shortly thereafter.
The end result looks like a slow burning memory leak. I've seen the worst cases take down servers every hour or two, but this can accumulate over time on a slower fuse as well.