News for hackers. Stuff that matters.
Re:NewsBlur: A feed reader with intelligence by boundlessdreamz on Tuesday October 26, 2010 @12:16PM (#1834779)
That's awesome. That's perfect tradeoff between the wishes of the parent website and convenience for your users.

I'm assuming you fetch feeds using a queue.

* How do you add jobs to the queue ? Do you run a cron every n minutes to check which feeds need fetching and add them to queue ?

* When you add jobs do you check to see if a previous job for fetching the queue is still in the queue or do you use flags to mark feeds for which the jobs are in queue and eliminate duplicate jobs that way ?

Re:NewsBlur: A feed reader with intelligence by conesus on Tuesday October 26, 2010 @12:22PM (#1834806)
Yes, I use celery/rabbitmq as a queue. But I don't bother checking for duplicates. If the feed is still in the queue by the time it comes back around for processing again, I let it run. That means that the queue is behind enough that it will even out in the wash.