* Postgresql and Oracle with skip locked
* MS SQL Server with readpast
* DB2 with skip locked data
If you use MySQL, it's going to be a bit more difficult.
This short screencast was also very helpful https://www.pgcasts.com/episodes/7/skip-locked/
Yes, Oracle's AQ product is built on top of an RDBMS. And yes, early versions of Microsoft's MSMQ used SQL Server. But most dev teams use the database as a shortcut and don't invest the effort in making their hacktastic DB queue work like a real product.
And you use "shortcut" like some kind of negative attribute. Of course they use DB as a shortcut, and they should - it works well and even has additional features (transactions with your primary store) that you wouldn't get with separate queue product.
On the other hand, you can grow solution for those organically over time to fit your needs.
In the end, I'd say they were equally reliable, but from a data analyst's perspective I very much missed the postgres based queue. I think the mechanics inherent to a database solution prompted the original implementers to keep messages around over time vs the mechanics inherent to an MQ solution where they became ephemeral (on the senders side.) Having access to those messages was a treasure trove for troubleshooting and for analytics. That and from a pure cost benefit stance, sending 10-20k messages a day to the ESP definitely didn't necessitate the expensive rearchitecture, as the DB solution was more than capable of handling the load on cheap hardware.
I also like the idea, but the counterpoint is: "now you have to monitor the queue".
(I only work in small apps for my own use so its more like Postgres and forget...)
(or, more often than not, because of the cost of doing this, you end up putting up with a slightly batshit-insane design...)
With careful management, it can be kept to a minimum, but I haven't seen that work for systems with a large numbers of developers working on it yet.
I'm exaggerating a little bit, but not too much (actually, on further reflection, I might actually be downplaying it a bit. some of our services schedule tasks with like 10 different services for every item processed, and we do tens of thousands a day).
Debugging issues in this mess is not fun, because there's so many places you need to check to see if it's the source of the failure or not, and a failure in one service could really be in a different service so you have to test all the way up and down the chain. For every bug.
But then, I'm mostly against microservices because they lead to harder problems on every place. Documentation isn't even the worst.
I feel like the article is attacking message brokering by discussing the disadvantages of bad use cases for them. A good use case for message brokers is when work needs to be done on an item, but not immediately.
My company uses them in a way I believe is quite effective. We pull in data from an external source, and send the ID of the item to about five different queues to do different tasks. Each time one of the queues finishes the work, it send a message to a validator that checks to see if all the work is done. If it isn't, it waits to get that message again from another worker. If it is, it marks that item as ready for the end user.
These are the kinds of use cases I think message brokers should be used for. Not to send a message and wait to get an answer back. Why not just use an HTTP request for that?
It doesn't matter, but it's more about debugging. If Service A does not work correctly, where is the bug? Is i Service A? Service B? The network?
The use at your company, is idiomatic to the paradigm. You have n different units of work, that can run seperately, so you do that and communicate with messages.
It's not that hard to do but it seems to take people by surprise and it's not helped by some poor defaults like almost everything in the RabbitMQ ecosystem silently blocking when the queue fills up (this probably happened transiently multiple times before it hit the level where it caused a visible outage but how many people will notice that if the default isn't to log or raise an error?).
There's a couple of nice ideas in here (the "message pump" for one at least) that I'm going to steal and use at work. It's also comforting to know that we're not completely crazy for using a DB to store tasks/processes that need to be run and retried if necessary instead of a message queue.
[1] https://en.wikipedia.org/wiki/Enterprise_Integration_Pattern...