It's actually not super complicated. A limit order book is a straightforward thing. If you can do FizzBuzz, you can probably whip up a toy implementation of a matching engine in an afternoon. People think it's complicated because of all the FUD (c.f. the comments on this story), but the basic mechanics of how a modern exchange operates are remarkably simple.
I'm referring to market microstructure in terms of the complex interactions amongst exchanges, the impact of Reg NMS, etc. Obviously the underlying algorithms can be boiled down to simple components.
For those interested in learning something about market microstructure, I highly recommend "Trading and Exchanges: Market Microstructure for Practitioners" as an introduction to the basics:
http://www.amazon.com/Trading-Exchanges-Market-Microstructur...
I've heard that the financial interconnections are remarkably similar to ecologies. Would be interesting to get some more of that tooling for analyzing ecologies like fisheries available to financial regulators.
That is fantastically complicated. As you say, anyone can write a matching engine.
The tediousness could be relieved by centralising market microstructure documentation. But given the frequency with which we change rules, it may make more sense to have an expanded class of lawyers or risk managers who are dedicated to understanding and promulgating the documentation.
I wouldn't go so far as to say the SEC, Finra, or the Fed Market Surveillance teams don't have a strong handle on microstructure - they have smart people. The SEC's job isn't to manually fix problems. It would be troubling if every time an exchange went down the SEC felt compelled to issue a new rule.
For instance, after NASDAQ went down for 3 hours on August 22, the SEC asked the NYSE and NASDAQ for a detailed timeline of events. Each exchange blamed the other. The SEC's response? It told the exchanges to cooperate and please fix the problem, declining to make any further public comment [1].
Trading at high speeds is like the wild west right now with little to no threat of law enforcement.
[1]http://uk.reuters.com/article/2013/08/27/uk-nasdaq-halt-nyse...