Re:Heroku Postgres: SQL Database-as-a-Service
by imbriaco on Tuesday November 22, 2011 @06:20PM
(#3267911)
You're exactly right about log streaming to S3. We open sourced the tool that we use to perform that: https://github.com/heroku/wal-e
[ Parent
| on HN ]
Re:Heroku Postgres: SQL Database-as-a-Service
by bdonlan on Tuesday November 22, 2011 @08:55PM
(#3268402)
S3 doesn't support continual logging - you need to decide on a file size up-front. Are you creating a new object for each log entry, or is there a window in which commits can be lost before the log gets flushed out, or do you have some other workaround...?
Re:Heroku Postgres: SQL Database-as-a-Service
by joevandyk on Wednesday November 23, 2011 @12:17AM
(#3268872)
There's a window of a couple minutes in which data could be lost (in the worst case scenario).
Re:Heroku Postgres: SQL Database-as-a-Service
by sector on Wednesday November 23, 2011 @12:30AM
(#3268895)
So it's postgres log shipping, with S3 as the target?
Re:Heroku Postgres: SQL Database-as-a-Service
by joevandyk on Wednesday November 23, 2011 @03:35AM
(#3269307)
Yup.
Re:Heroku Postgres: SQL Database-as-a-Service
by jaylevitt on Wednesday November 23, 2011 @04:42PM
(#3271871)
So would I need to (learn and) use PG9's replication with this? Or is one ephemeral-stored database plus shipped logs sufficient, assuming I can live with some downtime if the instance disappears?
As an experienced techie but a PG novice, what do I need to go learn about PG recovery procedures to safely use this?