Asynchronous Tasks with FastAPI and Celery

Re:Asynchronous Tasks with FastAPI and Celery by svvvy on Wednesday May 12, 2021 @02:05AM (#27127138)

I have found Celery to be a major pain. It's very complicated (Are your Celery tasks guaranteed at-most-once or at-least-once delivery? Are you sure?). There are bugs with the Redis broker that cause it to drop tasks: https://github.com/celery/kombu/issues/305

I heartily recommend: https://dramatiq.io/motivation.html - "if you’ve ever had to use Celery in anger, Dramatiq could be the tool for you".

[ Parent | on HN ]

Re:Asynchronous Tasks with FastAPI and Celery by staticautomatic on Wednesday May 12, 2021 @10:17AM (#27130603)

I love Dramatiq but I don’t get your point about task completion. You have to choose between at most and at least once.

[ Parent | on HN ]

Re:Asynchronous Tasks with FastAPI and Celery by nicolaslem on Wednesday May 12, 2021 @11:59AM (#27131814)

The main problem with Celery is that it supports many brokers and many features. This creates many subtle edge cases, specifically for brokers not built with AMQP, for instance the "visibility_timeout" attribute of the redis broker which feels like a hack.

This is why I created yet another task framework based on redis with robustness as a the main goal: https://github.com/NicolasLM/spinach

[ Parent | on HN ]

Re:Asynchronous Tasks with FastAPI and Celery by staticautomatic on Wednesday May 12, 2021 @07:55PM (#27136682)

Fair enough but I always interpreted the Celery docs as saying Redis was kind of mostly unofficially supported as a broker.

[ Parent | on HN ]