I am not advocating for this. I am simply demonstrating how the Disruptor leverages these kinds of techniques (memory barriers, et. al.) to provide a high performance outcome for its intended use cases.
I would be exhilarated if someone could demonstrate an even faster way to do what the Disruptor does (for x86 platforms).
If you need a total message order and many-to-all messaging it is indeed hard to do better than the Disruptor, but that's kind of a very specialized need. If you can relax the message ordering for example, multiple 1-to-all queues can be significantly more efficient.
I somewhat disagree with this. I have been able to build a lot of business systems that have nothing to do with fintech on top of the exact same abstraction. Order matching engines are not the only thing that can benefit from being able to deal with millions of serialized events per second. Most video games, especially multiplayer ones, can benefit from this style of abstraction too.
If the consequences of any given event may impact the processing of any subsequent event, then yes. For many ordinary systems this constraint holds true.
https://lmax-exchange.github.io/disruptor/disruptor.html
I think the Disruptor is probably still the most ideal approach for serializing multiple producers on an x86 platform. The core idea is to decouple the producer and consumer so that they do not contend over the same CAS variable (position in a ring buffer). If you only have a single producer, the CAS goes entirely uncontended and you will get tens to hundreds of millions of items per second without issue. Even with multiple producers, the figures are fairly incredible.
The trade off is that you basically have to sacrifice an entire core to busy-waiting if you want the world-class latency performance. Alternatives can yield or wait on conditions, but clearly would be slower.