Re:Introducing WAL-G: Faster Disaster Recovery for Postgres
by andruby on Sunday August 20, 2017 @03:53PM
(#15059965)
That's the usecase for archive_timeout. I set it to 60 seconds. So at most I'll have lost 60s + the time to transfer the file to s3, which shouldn't be more than a couple seconds.
[ Parent
| on HN ]
Re:Introducing WAL-G: Faster Disaster Recovery for Postgres
by ngrilly on Sunday August 20, 2017 @04:49PM
(#15060195)
According to PostgreSQL documentation, "archived files that are archived early due to a forced switch are still the same length as completely full files".
I'm afraid to use a lot of storage for WAL segments that are mostly empty:
16 MB per segment x 60 minutes x 24 hours x 7 days = 161 GB/week
Does WAL-G/WAL-E compression help?
Re:Introducing WAL-G: Faster Disaster Recovery for Postgres
by andruby on Tuesday August 22, 2017 @06:11AM
(#15070999)
Yes, the lzop compression helps a lot, and I imagine that mostly "empty files" will be more compressible.
On a staging server with little activity, the compressed WAL-E wal files go as low as 1.9MB per 10 minutes. (~2GB/week)
The production server has files between 4 and 12MB per 1 minute or less. (~220GB/week)
WAL-E has a good `wal-e delete retain` command that removes older base backups and wal files.