News for hackers. Stuff that matters.
Re:Post mortem on mastodon outage with 30k users (hachyderm) by cyberphobe on Sunday December 04, 2022 @01:20PM (#33856148)
These comments make me want to log off for a bit.

Post: we hit scaling issues caused by our failing disks and running image hosting and databases over NFS

HN: It’s obviously Ruby on Rails fault

Re:Post mortem on mastodon outage with 30k users (hachyderm) by kris-nova on Sunday December 04, 2022 @01:30PM (#33856232)
Hi. I wrote the post. Additionally I am responsible for operating Hachyderm (Ruby on Rails) and GitHub (Ruby on rails) for both my free-time and my day job.

I can say with certainty that Ruby specifically was not the bottleneck in our case. I do think that the rails paradigm can often lead to interdependent systems. We see this at GitHub and we also see this in Mastodon. Service A will do reads/writes against the same tables in the database that Service B also does. When service A is moved to an isolation zone, it can still impact Service B's performance.

In other words, I think any stateful framework with the flexibility that Ruby on Rails encourages bad behavior that can contribute to a noisy neighbor problem.

The point I am trying to drive home is that I agree. I can confidently say that Ruby on Rails is not the culprit in our case. To be honest I just ignore anyone who is quick to point fingers and assign blame either technically or personally.

Sorry hacker news got you down. If it helps my family and I are making Sunday morning pancakes with my puppy Björn today and we are all wishing you the best day ever.

Re:Post mortem on mastodon outage with 30k users (hachyderm) by cyberphobe on Sunday December 04, 2022 @01:32PM (#33856253)
aw thanks! I follow you on mastodon and it’s been great watching the updates as hachyderm grows
Re:Post mortem on mastodon outage with 30k users (hachyderm) by rbanffy on Sunday December 04, 2022 @02:00PM (#33856534)
Is there any good architectural breakdown of Mastodon software?
Re:Post mortem on mastodon outage with 30k users (hachyderm) by Kye on Sunday December 04, 2022 @02:09PM (#33856617)
It's surreal. I thought maybe I misread the post or something.