S3 is not the problem here. The problem is building a database on top of S3, and having to reimplement all the consistency, atomicity, transactions etc. on top.
>no thought to a schema, no migrations to manage
There is, in fact, always a schema. Some people choose to ignore it's there, to their detriment.
>Always avoid the fancy option when it makes sense.
It's not the 1980s. Postgres is not fancy, and Greenspunning it is a mistake.
>Almost guaranteed it's cheaper.
Cheaper than a 26-hour outage?
Most of those problems are moot if you're only ever writing from a single head node. If all your data is strictly ordered and you have no meaningful concurrency, this is a far, far simpler system.
Complex is Greenspunning a database and having it blow up in your face and cause a twenty-six hour outage. You never hear about such things with Postgres because Postgres is rock-solid.
But to your point, if your system requires less than a thousand lines of code to open a file, do basic parsing and processing (which no data storage system is going to do anyway), and write the output to another file, I personally can't say that Postgres or MySQL or any other solution is really worth the effort/cost to build and maintain. In the system being discussed, the benefits of an RDBMS simply don't matter: any strongly consistent key-value store would work.
> Complex is Greenspunning a database and having it blow up in your face and cause a twenty-six hour outage.
S3 didn't cause the outage, and from the look of it, neither did the code that processes the files. It was an application logic problem which caused issues during the restore process, and this would have been an issue regardless.
You could make an argument that the recovery being slower than it could have been was a problem, but it's wild to say that and imply that traditional databases have no performance cliffs. Especially when dealing with corruption or data recovery. Raw file storage will never have a Postgres transaction id wraparound incident (see: Sentry outage for most of a day in 2015, MailChimp/Mandrill for over a day in 2019) or have to rebuild a critical index.
In this case there was a hard coded concurrency limit with S3 of 250 outstanding requests. Bumping that up to 3000 would have been easy and reasonable (S3 rate limits at 5000). How confident would you be that your database can performantly handle a backfill during recovery? Have you provisioned enough iops? Are you running an RDS instance with only a burstable vCPU limit? To say Postgres is "rock-solid" (and make no mistake, I am a Postgres fanboy) dismisses the many and varying ways that it can fail in unusual and surprising ways.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
Did you miss where I said it's read-after-write strongly consistent?
AFAICT, using HN theres been roughly 30 hours of non-availability over 15 years. RDS didn't even support Postgres when Tarsnap was released.
EDIT: Tarsnap predates RDS.
Dumb is `aws s3 cp` and being done in 5 minutes.
He could have set up a more complex architecture and paid much more in hosting costs over the years to overengineer the solution. What would the benefits be? It might have avoided this one outage or saved a few hours restoring the data. The drawbacks? Much more time developing and maintaining the solution and higher subscription costs for users.
The solutions you are familiar with and comfortable with are perfectly valid. But you are falling into the trap of thinking "what I'm familiar with and comfortable with is the only valid answer and everyone else is wrong and stupid".
On a less technical note: Always avoid the fancy option when it makes sense. (From a veteran of building and maintaining large scale high performance high availability systems)