News for hackers. Stuff that matters.
Re:RQ – Simple Job Queues for Python by wojcikstefan on Friday January 03, 2020 @04:21AM (#21944453)
> Things like scheduled jobs can be done through sorted sets and persistent jobs are possible by shifting jobs into backup queues, but it is all a bit fragile.

What exactly is fragile about this approach? We've supported scheduled tasks in TaskTiger (https://github.com/closeio/tasktiger) via Redis's sorted sets and haven't had any issues.

Re:RQ – Simple Job Queues for Python by pselbert on Friday January 03, 2020 @05:50AM (#21944838)
> What exactly is fragile about this approach? We've supported scheduled tasks in TaskTiger (https://github.com/closeio/tasktiger) via Redis's sorted sets and haven't had any issues.

Using sorted sets for scheduled tasks isn't the fragile part, gluing it all together to prevent losing jobs by shifting into backup lists (or hashes) is the fragile part in my experience.

Re:RQ – Simple Job Queues for Python by notyourday on Friday January 03, 2020 @06:47AM (#21945103)
eval + multi-exec solves that as the issue
Re:RQ – Simple Job Queues for Python by pselbert on Friday January 03, 2020 @07:05AM (#21945192)
Sure, you can glue things together with lua. Here is a script that handles descheduling from a sorted set: https://github.com/sorentwo/kiq/blob/master/priv/scripts/des...

That definitely does the job. My point is that it is much more complex than a select/update clause in SQL.

Re:RQ – Simple Job Queues for Python by wojcikstefan on Friday January 03, 2020 @08:42AM (#21945718)
Yep, TaskTiger also uses Lua quite extensively ([0]) to ensure atomicity when moving tasks between various stages of processing (queued, active, scheduled, errored).

[0] https://github.com/closeio/tasktiger/blob/master/tasktiger/r...