News for hackers. Stuff that matters.
Re:PinLater: An asynchronous job execution system by bjt on Wednesday July 09, 2014 @07:42PM (#8012839)
I thought the same thing about Rabbit. PinLater's ACK system feels like a questionable reinvention of AMQP. I'm probably missing something.

Rabbit's advertised throughput doesn't reach 100,000/sec, but you should be able to shard your way around that. Or use Apache Kafka, which has the sharding built in.

Re:PinLater: An asynchronous job execution system by fintler on Wednesday July 09, 2014 @08:16PM (#8012931)
Just for comparison, LinkedIn is doing about 2.5 million messages per second (on average per day) with Apache Kafka.

http://www.slideshare.net/ToddPalino/enterprise-kafka-kafka-...

Re:PinLater: An asynchronous job execution system by hatred on Thursday July 10, 2014 @10:19AM (#8015040)
Precisely, this is the same query I had when I glanced down the post. It will be great if the author or someone from the team respond to the various trade-offs and why they did not go ahead with Kafka ?
Re:PinLater: An asynchronous job execution system by omrvp on Thursday July 10, 2014 @03:44PM (#8016775)
(I am the author of the blog post). I posted a reply below earlier this morning that hopefully addresses your question.
Re:PinLater: An asynchronous job execution system by bigtones on Wednesday July 09, 2014 @10:30PM (#8013279)
Looks like they needed to have built-in support for time based scheduled execution of jobs at a specific time in the future. Rabbit MQ does not have that (that I'm aware of) without doing some horrible hacks around one queue per message.
Re:PinLater: An asynchronous job execution system by kondro on Thursday July 10, 2014 @01:14AM (#8013732)
ActiveMQ's had that for a couple of versions though. Another highly performant/available, atomic, consistent, durable and mature queueing server.
Re:PinLater: An asynchronous job execution system by cyounkins on Thursday July 10, 2014 @11:02AM (#8015185)
Correct. Celery's implementation dequeues but doesn't acknowledge the task, holding it until the timer is reached.