News for hackers. Stuff that matters.
Re:Announcing RDS/Aurora Snapshot Export to S3 by etaioinshrdlu on Friday January 24, 2020 @02:19PM (#22141513)
I use rds Aurora but i run a daily cron job to export the entire thing with mysqldump + gzip to s3.

I pass some flags to mysqldump to avoid locking the db and otherwise interfering with production. It also dumps from the reader node not the writer.

I also clear some tables and restore it daily to a development DB.

Sadly this looks like it only supports parquet, a rather usual db dump format. Even if, I'm sure, it's way more efficient to process using some modern tools than a text sql dump.

I just like open formats and interoperability and AWS's rds offerings are always just slightly 'off'.

Re:Announcing RDS/Aurora Snapshot Export to S3 by sudhirj on Friday January 24, 2020 @03:13PM (#22141984)
Parquet is an open format. It’s part of the Apache foundation.

Would have preferred CSV as well, though, easier to work with. There’s also a little known feature called S3 Select that allows you to run SQL on S3 files and selectively retrieve data, which is what Parquet is for in the first place.

Re:Announcing RDS/Aurora Snapshot Export to S3 by etaioinshrdlu on Friday January 24, 2020 @05:05PM (#22142900)
Can you load that dump using any open source tools outside of AWS and recreate the database identically? Because you can do that with mysqldump.
Re:Announcing RDS/Aurora Snapshot Export to S3 by derision on Friday January 24, 2020 @07:00PM (#22143529)
I don't think this is meant for recreating the database, but rather exporting large sets for analysis somewhere else. But as far as tools yes, parquet can be parsed by any tool that knows the format. It's not too uncommon