We’ve run into various bugs and weird performance gotchas (like the workers prefetch jobs which is terrible if they aren’t all the same size)
I'm sure if I really tried, I can get celery to be very reliable... but I never really got there.
Also for whatever reason I have NEVER been able to get celery to be reliable for its scheduling/cron stuff. It just starts to fail. I use this library for that, which I have never had problems with: https://schedule.readthedocs.io/en/stable/
I've been using Celery for a long time in production now. Nothing crazy and it's a fairly basic set up of "execute job, thanks!" along with a beat server. Over the last 6-7 years an off the top of my head guess would be that there's been at least 5 million jobs processed. It's not huge volume when measured over years but it's been stable.
One server was running for 6 months without being updated. That's a Celery worker process running for ~180 days uninterrupted. It served hundreds of thousands of jobs without maintenance. A lot of these were pretty beefy tasks too like performing HTTP requests that got 400mb XML responses and then parsed them.
It applies to Django, Flask or any Python system using it. All of it still applies today.
It covers a few use cases on the before vs after of using Celery and touches base on why I'd consider using Celery over other solutions such as async / await. The TL;DR is Celery brings a lot to the table around tracking and retrying jobs. It's also nice to separate your web and worker workloads into different processes since they have much different requirements usually.