News for hackers. Stuff that matters.
Re:Scaling Mastodon in the Face of an Exodus by pmarreck on Friday November 11, 2022 @01:05PM (#33564344)
> TL;DR: Mastodon’s Sidekiq deferred execution jobs are the limiting factor for scaling federated traffic in a single-server or small cluster deployment. Sidekiq performance scales poorly under a single process model, and can be limited by database performance in a deployment of the default Dockerized configuration.

Well, that in short explains why Pleroma/Elixir is a much better stack to build this in. You don't need Sidekiq (https://sipsandbits.com/2020/08/07/do-we-need-background-job...) and you aren't stuck with a GIL (Global Interpreter Lock).

Re:Scaling Mastodon in the Face of an Exodus by sjustinas on Friday November 11, 2022 @02:48PM (#33565835)
Just to note, Pleroma also uses a job queue: Oban. https://hexdocs.pm/oban/Oban.html
Re:Scaling Mastodon in the Face of an Exodus by pmarreck on Friday November 11, 2022 @03:36PM (#33566439)
Oban is great. (I subscribe to Oban Pro.) It's also seemingly very superior to Sidekiq-like solutions for job management.

But you don't need that. It's certainly a nice-to-have.

Re:Scaling Mastodon in the Face of an Exodus by krainboltgreene on Friday November 11, 2022 @02:56PM (#33565960)
> Well, that in short explains why Pleroma/Elixir is a much better stack to build this in

As someone who has extensive experience in both elixir and ruby, and mastodon itself, no this is doesn't.

Re:Scaling Mastodon in the Face of an Exodus by pmarreck on Friday November 11, 2022 @03:39PM (#33566490)
I read that a Pleroma instance can handle at least 10-50x the traffic of a Mastodon instance running on the same hardware. Based on the reports from other sites that have switched from Ruby to Elixir, this rings true. Are you claiming otherwise? I mean, the lack of a GIL alone (thank you, functional language with immutable data!) should give a pretty massive performance improvement
Re:Scaling Mastodon in the Face of an Exodus by faraaz98 on Friday November 11, 2022 @10:49PM (#33569768)
I like Elixir but as of the latest Ruby version, you can use ractors. They aren't bound by the gil
Re:Scaling Mastodon in the Face of an Exodus by pmarreck on Saturday November 12, 2022 @07:42AM (#33572499)
Did not hear of them. Interesting. Also requires a code rewrite to take advantage of them, and still considered very experimental.