News for hackers. Stuff that matters.
Re:Approaches to PostgreSQL Replication and Backup by colanderman on Friday February 23, 2018 @09:19PM (#16451245)
A fourth option, for those in physical datacenters who do not need to run active/active, is of course to eschew network replication entirely in favor of a RAIDed JBOD shared between two or more servers. If you have the hardware, this is often the easiest to configure, and the most efficient.

EDIT: I should clarify that, like the other HA solutions in the article, of course you need to couple this with a backup solution, like periodic dumps to S3, such as the article suggests.

Re:Approaches to PostgreSQL Replication and Backup by colanderman on Friday February 23, 2018 @10:34PM (#16451495)
Those downvoting, care to give a technical reason why I am wrong?
Re:Approaches to PostgreSQL Replication and Backup by koolba on Friday February 23, 2018 @10:48PM (#16451533)
The short answer is that RAID is not a backup solution.
Re:Approaches to PostgreSQL Replication and Backup by rosser on Friday February 23, 2018 @10:53PM (#16451553)
It also doesn't manage failover. Shared storage is a totally viable approach. (I've worked as a PostgreSQL DBA for over a decade and have built multiple shared storage-based HA environments for multiple employers.)

If you mis-manage failover, you will destroy your data. This makes proper backup even more important.

Re:Approaches to PostgreSQL Replication and Backup by colanderman on Friday February 23, 2018 @11:02PM (#16451590)
Nor does Postgres replication manage failover. Either way you need a 3rd party tool.

Yep, botched failovers are bad. Of course, a botched failover between replicated DBs is no walk in the park either. In many such cases you'll still want to pull backups anyway, if there's no clear way in your domain to merge two "authoritative" databases.

Re:Approaches to PostgreSQL Replication and Backup by rosser on Friday February 23, 2018 @11:57PM (#16451766)
> In many such cases you'll still want to pull backups anyway...

In all cases where you care about your data, you must take backups. There is no substitute for that step.

Additionally, you want to periodically restore your backups and test them for validity — ideally automatically. There's a saying in the ops world, borne of bitter experience:

"Your backups are only as good as their last successful restore."

Re:Approaches to PostgreSQL Replication and Backup by colanderman on Saturday February 24, 2018 @10:15AM (#16453545)
What grantwu said. You misinterpreted me. Seems to be a theme today. I guess I should spell things out more clearly in the future.
Re:Approaches to PostgreSQL Replication and Backup by rosser on Saturday February 24, 2018 @05:19PM (#16455775)
Ok, my apologies. I've never heard that phrasing before, but now that you clarify, I understand it.

Ambiguity in online discourse is an ongoing problem. Thanks for helping resolve this occurrence without acrimony.

Re:Approaches to PostgreSQL Replication and Backup by grantwu on Saturday February 24, 2018 @02:30AM (#16452210)
I read "pull backups" as "restore from backup" not "make backups".
Re:Approaches to PostgreSQL Replication and Backup by colanderman on Friday February 23, 2018 @10:49PM (#16451541)
I didn't say it was? Nor is replication.

But thanks for the reply, I've edited to clarify that disk sharing is an alternative to the three HA solutions in the article, not the one backup solution (which in all examples is the mundane "copy to S3").

Re:Approaches to PostgreSQL Replication and Backup by rosser on Friday February 23, 2018 @11:01PM (#16451584)
The Fine Article is, indeed, light on meaningful approaches "backup", despite its prominent presence in its title.
Re:Approaches to PostgreSQL Replication and Backup by colanderman on Friday February 23, 2018 @11:32PM (#16451676)
It's also light on approaches to HA (hence my – apparentely misunderstood – addition to the list). Postgres's documentation itself is much more thorough: https://www.postgresql.org/docs/10/static/different-replicat...

And gives a little more insight into backup options too: https://www.postgresql.org/docs/10/static/backup.html

Re:Approaches to PostgreSQL Replication and Backup by cuu508 on Saturday February 24, 2018 @04:13AM (#16452454)
Sorry for the dumb question, but which of the 3 approaches is pg_basebackup?

My understanding was/is that with pg_basebackup the new replica would only need to replay state changes since the last checkpoint, not "the entire state from the primary node—from the beginning of time". What am I missing?

Re:Approaches to PostgreSQL Replication and Backup by koolba on Saturday February 24, 2018 @06:57AM (#16452905)
Definitely the third one and possibly the first one depending on what’s being backed up. It seems to imply it’s just a logical pg_dump backup going to the offsite store but doesn’t clarify.

The second option is at the physical block layer. It’s not specific to any one DB but is a bit of a blunt instrument (works everywhere but lacks finesse).

The third option would involve base backups being pushed off site as well as archived WAL logs. The replicas would restore the base backups, then catch up via WAL logs. Finally you can directly link up to the master to have a live standby for read only queries. This is the approach used by the WAL-E and WAL-G backup/restore tools and is the one I generally recommend as it has the most flexibility and resilience.

Re:Approaches to PostgreSQL Replication and Backup by skewedlines on Sunday February 25, 2018 @02:05AM (#16457606)
Hmm, just a quick clarification for the first approach. It says that the replay operation may then introduce significant load on the primary node. Is this because when new nodes are added, the logs have to be dumped to the new slave nodes from the beginning, and the IO to do so is expensive, possibly interfering with normal operation of the master node?
Re:Approaches to PostgreSQL Replication and Backup by mixmastamyk on Monday February 26, 2018 @01:47AM (#16463617)
That’s how I read it.