Re:Announcing DJJob, a PHP port of delayed_job
by aonic on Friday October 22, 2010 @08:58AM
(#1819481)
Not sure who your question was directed to, but I've implemented a similar system in the past for a website with 10m+ users and 15+ web servers. It scaled just fine, we were sending close to 150K emails a day, among other background jobs for reporting, data cleanup, automated spam checks, etc
In the interest of some outside figures, how have you found this to scale? I implemented a similar solution as more of a stopgap until I had the time to get RabbitMQ installed with a "proper" worker dispatcher (more moving parts but less stress on the DB). I mostly ask because we implemented it for our notification system where we had to generate a send large batches of custom emails which was bogging down the page (and hard to debug where in the chain we were getting failed messages).