I'm not saying the trade-off is bad or good, but there is a cost; it's not free.
> If you're already using a tool
Often, I'm not.
> illusion of readability
That depends entirely on the protocol design; some are better than others. Binary protocol can also be a mess. More importantly, binary protocols will always require external definitions, while some text protocols do not.
> bug-ridden and rife with security holes
Changing from readable tokens and ASCIIified intege4rs to packed binary doesn't magically fix bus or security holes.
> what do I need to do for compatibility
That's easy: use the robustness principle[1].
(and most RFC-based protocols say you should send a CRLF).
A possible argument against this could...
> how long could this line be
...also be an argument against length fields. If you want reliable parsing - a very good idea - a good argument[2] can be made to only use context-free protocols (i.e. use parser-combinators).
[1] https://en.wikipedia.org/wiki/Robustness_principle
[2] https://media.ccc.de/v/28c3-4763-en-the_science_of_insecurit...
Meredith covers this specific issue at 33:25 in her 28c3 talk I referenced ([2]). "Be definite in what you accept" can emulate traditional "liberal" parsers if the compatibility is strictly defined in the grammar.
I really should have phrased that better; the robustness principle is great historically, and is therefor necessary in many current protocols, but newer protocols should be using well-defined either regular grammars or deterministic context-free grammars. As Meredith and Sergey explain in that talk, moving away form Turing complete protocols would prevent an important class of exploits.
They don't, and the readability is a feature.
I'm not sure how you conflate XML and JSON as synonymous to "protocol" and also requiring tools to filter, but than imply that a "good tag/length/value" format would somehow be superior to the former.
Because, well except for readability and ease of debug, that you lose completely by going binary, and message size that improves by going binary, I can't find any other difference.
An honest question: Is it possible that they suck for you and people like you, skilled, experienced engineers with sophisticated tools, and with much less time than money - but they don't suck for less experienced, less equipped, hackers or tinkerers with time to try things but little money or expertise?
Remember that those people built quite a few things, from Facebook to Napster to Dell to the web browser to the web itself (just off the top of my head). Empowering them is arguably the key to innovation.
Also, people don't seem to complain about the cost of HTTP over SSL in these discussions on HTTP/2. I've never understood that. If you're allowing for TLS, why not HTTP/2?
ASCII has a much lower barrier to entry. By the same reasoning, everything is 1's and 0's so everything is the same - just use better tools.