I'm sure my information is old, perhaps these things are improved in newer versions. Still, the behavior of the Erlang VM itself was relevant.
As a C++ programmer, I wouldn't feel comfortable abstracting away memory management for this type of server which is both so memory intensive and requires high reliability.
But of course many people swear by highly reliable JVM based stuff, so perhaps I'm wrong about it. :-)
It was true that previously RabbitMQ could run out of memory and be unable to take more messages, but that changed in 2.0 and it will now swap messages to disk. Of course, then you pay the consequences of slow disks and can always run out of disk space, but nothing is invincible.
Prior to 2.0, the contents of Rabbit's queues were always in RAM. Even durable queues with persistent messages always had their contents mirrored in RAM. As of 2.0, Rabbit can store queue contents on disk or in RAM, so as it faces memory pressure, it will send data to disk. If the disk is too slow for it to relieve its pressure, it will block connections that have sent data in the past until it can free up enough memory to continue.
Last I checked, Rabbit does by default keep the associative map between queues and message in RAM, so its memory usage isn't technically bounded (it's a around a dozen bytes per message, IIRC). If that's a problem, the toke plugin uses tokyo cabinet to store the associations, which I believe allows Rabbit to have entirely bounded memory use. I also haven't checked on that for a while, so it may be that toke is installed by default now.
Erlang was created by a telco in the 1990s and since then has been battered in production by serious users. It's real software. The version that Ericsson provide as open source is not crippleware as you seem to imply.
"RabbitMQ pays the performance cost of bouncing everything off the disk"
No it doesn't.
RabbitMQ only uses the disk if tell it to do so, eg if you require messages to be persisted when they cannot be delivered immediately.
"memory usage can grow and grow"
If you stuff data into a messaging server without draining it on the consumer side, then memory usage will grow. This will also happen if you write your messaging system in C++.
There are two solutions to this problem: - flow control, where you tell producers to back off - paging to disk, where you flush data from memory when it is on the disk
Neither of these is trivial to implement which is why there is a big gap between toy messaging systems and serious products.
As others point out on this page, RabbitMQ has support for both these features. In particular a lot of memory management capability has been added since 2.0.
I hope this helps.
No matter what your queue provider, you're most likely just using TCP sockets anyway, so don't think you need to be using Erlang just to take advantage of RabbitMQ.
Yes, this means the MQ can basically be thought of as a black box. But that's also kind of the point.