News for hackers. Stuff that matters.
Re:Postgres Incremental Backup by MuffinFlavored on Thursday January 11, 2024 @10:42PM (#38963397)
I wonder what stuff like this means in terms of electrical impact around the world at scale.

I'm being dramatic obviously but like... less CPU cycles, less network bandwidth, etc. etc.

If adopted by top 100 "large" database/operations, is it as much as like 100 people giving up their gas/petrol/diesel car?

Re:Postgres Incremental Backup by peterfirefly on Thursday January 11, 2024 @10:45PM (#38963416)
Re:Postgres Incremental Backup by flockonus on Friday January 12, 2024 @12:47AM (#38964421)
Glad I've learned about this fascinating effect, but it doesn't seem reasonable that such a niche improvement on Postgresql backup would have an impact on the price of electricity or another resource.
Re:Postgres Incremental Backup by fastball on Friday January 12, 2024 @01:10AM (#38964555)
No, but you could imagine a situation where a feature added to a database makes people more likely to use that database, even in places where maybe a "lighter weight" database (e.g. SQLite) would've been good enough, therefore increasing electricity usage overall.
Re:Postgres Incremental Backup by peterfirefly on Friday January 12, 2024 @05:37AM (#38966414)
That + many more people will take backups (and most of those will be unnecessary).
Re:Postgres Incremental Backup by flockonus on Friday January 12, 2024 @03:02PM (#38973206)
Still, imho, hardly the scale to have the effect visible, if you think how many servers exist vs. the electricity demand reduction.

One example I'd find reasonable, perhaps the Apple Silicon M family has a shot at it, when comparing the energy draw diff (less than half of Intel's equivalent) multiply by the adoption at scale.. maybe that.

One more straightforward that definitely have a chance at achieving the effect: LED lightbulbs: huge consumption reduction, huge scale.

Re:Postgres Incremental Backup by devnonymous on Thursday January 11, 2024 @11:56PM (#38963949)
I'm willing to bet that beyond a certain threshold there's no use for an incremental backup. You either would use (delayed) replication[1] to have an 'incremental' copy or would take full backups. I don't think any large database users rushing to use incremental backups when they have full redundancy.

[1] for example https://about.gitlab.com/blog/2019/02/13/delayed-replication...

Re:Postgres Incremental Backup by rhaas on Friday January 12, 2024 @08:38AM (#38967760)
I agree that for many use cases, standbys and in particular time-delayed standbys are a better option than backups. That said, having off-line backups in tamper-proof storage is a good call, too.
Re:Postgres Incremental Backup by fbdab103 on Friday January 12, 2024 @01:45AM (#38964807)
I think about this every time I see my corporate laptop spinning up because of bloated malware detection software, Teams, OneDrive, etc all seemingly unoptimized and burning resources across hundreds of millions of machines worldwide.
Re:Postgres Incremental Backup by mosselman on Friday January 12, 2024 @02:03AM (#38964960)
I did a thought experiment with made up numbers. I just woke up so forgive me if my numbers don’t add up. At least they can serve as ballpark numbers.

Let’s say a full backup takes 3 hours and an incremental one 3 minutes.

Let’s say I want at most 3 hours of lag in my backups. That would mean 3 hours every 3 hours. That is about 720 hours per month for a full backup or 720 minutes (12 hours) for incremental backups. Let’s say we Still do a full one once a day then that is still 102 hours per month in total.

Even if you do 1 incremental per hour you’d end up at 36 hours a month. If 3 minutes were true for the every 3 hour scenario, the every 1 hour scenario would also mean shorter backups so maybe 1 minute, which adds up to 12 hours.

All this to say that it might be more efficient.

Re:Postgres Incremental Backup by scoot on Friday January 12, 2024 @03:35AM (#38965570)
> Let’s say I want at most 3 hours of lag

AKA Recovery Point Objective (RPO)