News for hackers. Stuff that matters.
Re:Heroku Postgres: SQL Database-as-a-Service by notaddicted on Tuesday November 22, 2011 @04:21PM (#3267425)
Considering their "Continuous Protection" feature, could they be streaming a write log out to S3? I think that way they could use the backup as the sole guarantee of durability, and use instance storage for the database.

EDIT: also, I'm not sure if this is generally known elsewhere, but might they not be on ec2, they are advertising 1.7 GB cache, which is the same as a small instance, except without any room for the operating system or postgres itself.

Re:Heroku Postgres: SQL Database-as-a-Service by imbriaco on Tuesday November 22, 2011 @06:20PM (#3267911)
You're exactly right about log streaming to S3. We open sourced the tool that we use to perform that: https://github.com/heroku/wal-e
Re:Heroku Postgres: SQL Database-as-a-Service by bdonlan on Tuesday November 22, 2011 @08:55PM (#3268402)
S3 doesn't support continual logging - you need to decide on a file size up-front. Are you creating a new object for each log entry, or is there a window in which commits can be lost before the log gets flushed out, or do you have some other workaround...?
Re:Heroku Postgres: SQL Database-as-a-Service by joevandyk on Wednesday November 23, 2011 @12:17AM (#3268872)
There's a window of a couple minutes in which data could be lost (in the worst case scenario).
Re:Heroku Postgres: SQL Database-as-a-Service by sector on Wednesday November 23, 2011 @12:30AM (#3268895)
So it's postgres log shipping, with S3 as the target?
Re:Heroku Postgres: SQL Database-as-a-Service by joevandyk on Wednesday November 23, 2011 @03:35AM (#3269307)
Yup.
Re:Heroku Postgres: SQL Database-as-a-Service by jaylevitt on Wednesday November 23, 2011 @04:42PM (#3271871)
So would I need to (learn and) use PG9's replication with this? Or is one ephemeral-stored database plus shipped logs sufficient, assuming I can live with some downtime if the instance disappears?

As an experienced techie but a PG novice, what do I need to go learn about PG recovery procedures to safely use this?