(Honest question; I have no direct experience with multicast.)
Most TCP implementations handles it well and Linux has well written guides on how to tune parameters when the defaults don't match well to your specific situation well. Multicast just fails catastrophically in arcane ways that depend upon your specific hardware, network topology and mix of your firmware version on your switch/routers. It makes it very hard to "design" a product around that kind of fragile system.
Wouldn't traditional Paxos already be dead in the water at that point?
Also regarding the point about multicast storms -- I was imagining something like <100 nodes in a multicast group -- is that naive? What obvious use case am I missing? I was basically thinking of this scheme in front of a database layer
Anecdotally, non-multicast setups are easier to scale in hardware (because you can push commodity farther) and most critically are easier for the normal type of engineer you'd find in the tech sector to reason about (because many aren't comfortable with networking equipment, so it's much easier moving that state out of the network and into layers they can easily debug).
At any rate, most cloud SDNs don't support IP multicast (unless you use an overlay like Weave), so it's sort of a nonstarter for them.
These days I might use Redis pub/sub along with a multi-level cache (e.g. LMDB + local Redis + remote Redis) if I need to distribute data to a large number of clients efficiently.
You can, of course, use Rendezvous without multicast.
And: does anyone still use it? The last exchange I worked with that did was migrating from it to JMS (which sort of gives lie to the idea that Rendezvous's multicast performance was that big a deal).
Now that 10Gb+ is commonplace and modern servers can service hundreds or thousands of message bus consumers with efficient kernel-managed event loops, yes, it's arguably less needed now. But nobody will claim it's efficient from a network perspective. :)
Is well regarded & open source
https://www.nasdaqtrader.com/content/technicalsupport/specif...
It is actually quite good.
FWIW when you talk about different requirements, I'm not sure you're correct. For traders, sure, oftentimes they can just dump old data if they fall behind because it's mostly irrelevant. However, stock exchanges have performance and reliability/availability as top requirements. They must record all transactions and can't crash or lock-up. Most use multicast internally and externally. They would never be able to scale and maintain a low response time with TCP.
In a dynamic environment where you'd be scaling continuously and adding components and servers and building new services, this becomes a non-starter. Multicast implementations doesn't fail gracefully -- there's very little warning before things come to a complete halt and need complete shutdown and reboot of entire network.
When IGMP Snooping is enabled, the switches themselves will limit multicast messages only to ports that have actually joined a group, which prevents multicast storms (if I'm understanding you correctly) in the first place.
In the kind of environments I worked in, this was a non-starter.
edit: also "limit multicast messages only to ports that have actually joined a group" part is usually implemented by bloom-filters that I talked about above. They work well as long as there are not many groups and not many members.
I would be curious in your experiences if you're able to share it, but do very much believe there are options other than software for implementing multicast filtering on switches. What type of environments that you worked in made IGMP Snooping a non-starter?
1. As implemented by most (read: all) network hw, it works fine when congestion is low. As soon as congestion starts increasing, the packet loss amplifies.
2. If you have large multicast groups, the bloom filters used for membership test become very full, hence false positive rate rapidly increases. This leads to multicast storms.
In my experience, anything which is not TCP (or, to lesser degree UDP) is a non-starter for building reliable distributed systems. Despite the claims of "lower overhead" the TCP protocol stack gets most love and hence has been optimized heavily on every significant operating system.