News for hackers. Stuff that matters.
Re:The State of Vacuum in Postgres by hinkley on Friday January 05, 2018 @03:48PM (#16081505)
A number of GC and other resource reclamation systems, especially soft real-time ones, have worked on accounting the cleanup costs to the parts of the system that make the mess in the first place.

In these systems, a function that is very careful with resource allocation will almost never experience a significant slowdown due to the background processing, because they either pay no cost or an amortized cost. For instance, one collector I knew of would garbage collect up to 10 objects on every allocation. If large allocations are front loaded then a full cycle almost never happens. In Swift or C++ or Rust you pay for your allocations as soon as they go out of scope.

In the context of an MVCC system, running a long query causes considerably more versioned data to pile up. If part of the cost of retiring that data was payed at the end of the transaction, several things happen. One, other transactions are less impacted by the overhead. Two, you end up throttling the bad actors, which encourages the problem to be fixed, and in the meantime the high water mark for vacuuming is reduced.

Re:The State of Vacuum in Postgres by rosser on Friday January 05, 2018 @03:52PM (#16081525)
Are you legit suggesting tying the cost of GC to transaction commit in a database platform that is very commonly used for OLTP workloads?
Re:The State of Vacuum in Postgres by hinkley on Friday January 05, 2018 @04:05PM (#16081629)
Those are exactly the systems that are falling over already, aren’t they? They’re already tied together whether that’s inconvenient or not. Right now (or at least, recently) the performance simply falls off a cliff instead of degrading.

A lot of the tools we have for dealing with down services also can deal with degraded ones. If for instance throttling votes per second keeps the system usable for everyone else then that has been done before and can be done again. But you have to do it before the database has a seizure, not after.

[edit: I’m talking about back pressure. This is not a new or crazy concept. Your incredulity and $5 will buy me a cup of coffee]

Re:The State of Vacuum in Postgres by rosser on Friday January 05, 2018 @04:12PM (#16081687)
I wouldn't characterize things as "falling over", except perhaps in cases of operator incompetence.

There's certainly room for improvement — The Fine Article wouldn't exist otherwise — and backpressure might be worth exploring as a means of improving things. If that backpressure causes the system to slow down in a way that adversely affects customer or end-user interactions, however, it will often just be turned off, mooting it — or making things worse.

Re:The State of Vacuum in Postgres by hinkley on Friday January 05, 2018 @06:21PM (#16082659)
Like a lot of people, I was barely aware of VACUUMing before several high profile OLTP systems had unplanned outages because Postgres' requests per second dropped off dramatically and suddenly due to this arcane form of resource exhaustion. That's a performance cliff. I didn't mean it is hyperbole but as jargon. System hits a bottleneck, rate-of-change alters drastically/unmanageably.
Re:The State of Vacuum in Postgres by rosser on Friday January 05, 2018 @08:30PM (#16083281)
That sounds like operator error to me. If you (the general "you") use a database in production without knowing what sorts of maintenance it requires, that's neither the database's nor its authors' fault.

I guess you could argue the tool is somehow "flawed" for needing maintenance. I'd counter with the invitation to show me tools that don't.

Re:The State of Vacuum in Postgres by quickben on Friday January 05, 2018 @05:02PM (#16082120)
new and delete are fast.

What's slow are stuff like java GC Eden management where deletes are an afterthought to what is a time-stamped LRU reference management server, and a lot more.

So, no. I don't think you can compare C++ with systems that run full fledged GCs and threat them in the same bullpark.