News for hackers. Stuff that matters.
Re:PostgreSQL 17 by ksec on Thursday September 26, 2024 @10:54AM (#41659104)
With 17, is Vacuum largely a solved issue?
Re:PostgreSQL 17 by ses1984 on Thursday September 26, 2024 @11:04AM (#41659223)
I’m not up to date on the recent changes, but problems we had with vacuum were more computation and iops related than memory related.

Basically in a database with a lot of creation/deletion, database activity can outrun the vacuum, leading to out of storage errors, lock contention, etc

In order to keep throughput up, we had to throttle things manually on the input side, to allow vacuum to complete. Otherwise throughput would eventually drop to zero.

Re:PostgreSQL 17 by sgarland on Thursday September 26, 2024 @11:11AM (#41659304)
You can also tune various [auto]vacuum settings that can dramatically increase the amount of work done in each cycle.

I’m not discounting your experience as anything is possible, but I’ve never had to throttle writes, even on large clusters with hundreds of thousands of QPS.

Re:PostgreSQL 17 by jabl on Thursday September 26, 2024 @12:52PM (#41660624)
There was a big project to re-architect the low level storage system to something that isn't dependent on vacuuming, called zheap. Unfortunately it seems to have stalled and nobody seems to be working on it anymore? I keep scanning the release notes for each new pgsql version, but no dice.
Re:PostgreSQL 17 by qazxcvbnm on Thursday September 26, 2024 @01:26PM (#41660967)
I think OrioleDB is picking up from where zheap left things, and seems quite active in upstreaming their work, you might be interested to check it out.
Re:PostgreSQL 17 by imbradn on Thursday September 26, 2024 @01:42PM (#41661108)
No, vacuum issues are not solved. This will reduce the amount of scanning needed in many cases when vacuuming indexes. It will mean more efficient vacuum and quicker vacuums which will help in a lot of cases.