Re:Scaling Mastodon with systemd template units
by ratorx on Tuesday November 29, 2022 @08:43PM
(#33795871)
Bit OT: Are dynamically scalable process workers on a single machine the right choice?
It seems a bit niche to be in a situation where you need the flexibility of dynamic scheduling whilst only having a single machine. Once you have more than 1 machine, systemd isn’t designed for it anyway, so you’d be looking at a cluster scheduler like Nomad or Kubernetes anyway.
In the single machine case, would it not be better to either:
a) Just spin up a static number n of workers (where n is the number of machine cores) and let them idle if not in use.
b) Use a thread pool (or green threads) inside a single process with shared memory (if e.g. idle memory use by workers is a concern).
I can’t immediately see the benefit of being able to have dynamic control over a number of workers AND keep them as separate processes that need to be adjusted manually.
I guess there may not be a choice if the software is not designed to be run inside threads AND memory use is a concern.
It seems a bit niche to be in a situation where you need the flexibility of dynamic scheduling whilst only having a single machine. Once you have more than 1 machine, systemd isn’t designed for it anyway, so you’d be looking at a cluster scheduler like Nomad or Kubernetes anyway.
In the single machine case, would it not be better to either:
a) Just spin up a static number n of workers (where n is the number of machine cores) and let them idle if not in use.
b) Use a thread pool (or green threads) inside a single process with shared memory (if e.g. idle memory use by workers is a concern).
I can’t immediately see the benefit of being able to have dynamic control over a number of workers AND keep them as separate processes that need to be adjusted manually.
I guess there may not be a choice if the software is not designed to be run inside threads AND memory use is a concern.