News for hackers. Stuff that matters.
Re:Multi-Tenant Queues in Postgres by ucarion on Thursday April 18, 2024 @04:21PM (#40080309)
This is pretty fancy stuff! Sorry if I'm just not reading carefully enough, but does this approach account for tenants whose messages take longer to process, as opposed to a tenant that sends a larger volume of messages?
Re:Multi-Tenant Queues in Postgres by abelanger on Thursday April 18, 2024 @04:48PM (#40080562)
No, this assumes you've already split your tasks into quanta of approximately the same duration - so all tasks are weighted equally in terms of execution time. If each of the tasks have different weights you might be looking at something like deficit round-robin [1], which I've looked at implementations for in RabbitMQ and we're thinking about how to implement in PG as well [2].

[1] https://www.cs.bu.edu/fac/matta/Teaching/cs655-papers/DRR.pd...

[2] https://nithril.github.io/amqp/2015/07/05/fair-consuming-wit...

Re:Multi-Tenant Queues in Postgres by ucarion on Thursday April 18, 2024 @09:02PM (#40082416)
Makes sense!

My gut tells me that it would often make sense to jump straight to shuffle sharding, where you'd converge on fair solutions dynamically, in a lot of cases. I'm looking forward to that follow-on article!