(I know there are good reasons for this involving the effort required and maintenance etc. it just bugs me.)
In Windows, PowerShell is a very good example of a low-ish-level (composable commandline apps and snippets) programming environment that isn't shackled by that paradigm.
As others have pointed out, text-based serialization has significant overhead. To save a few hours of debugging, you're willing to spend gigabytes of bandwidth, an uncountable number of CPU cycles and memory, and all the associated electricity and hosting costs over the lifetime of your service? Penny wise and pound foolish, to say the least.
Text-based serialization formats are just plain stupid for any type of service that handles over 100 requests per second. A good developer is not hampered in the slightest by the fact that their serialization format isn't text.
Would I switch to a more computationally expensive format to save a few hours debugging each time someone makes substantial changes to my application's message format? Very possibly.
Would I switch to a more computationally expensive format to prevent bugs (the easier it is to view, or remember how to view, what your code is sending/receiving, the less likely it is that lazy people will skip bug-hunting/QA steps that would show it)? Almost certainly.
Should you use a binary format if you're sending tons of data per second? Well, it depends. It's like microservices: if you have the tooling to make dealing with that format from the dev/debug/tracing side a breeze, go for it; using binary provides substantial savings, as you point out. If you don't have the tooling/time to make it as easy as text to introspect, and I mean that in the most absolute way, there might still be persuasive reasons to switch, but know that if you do, you are also buying a lot of "debuggability debt".
Readable plain-text protocols remove a constant overhead from all debugging/introspection tasks. If you use, say protobufs, you'll still have to convert back into strings to see what was in individual messages. This makes reading, for example, logfiles or wireshark captures with "grep" more difficult. Sure, there are tools that you can throw before "| grep" that convert to text, but that's one more piece of cognitive/debugging overhead you have to remember, and one more thing that new developers have to know to do if they're used to the standard unix/text-oriented way.
That benefit applies even if your messages (or logs of messages) are stored/viewed in a database that speaks your binary protocol and provides tools to search/see the data as text. Whether or not you have something like that, it will always (always) be routinely necessary, or at least incredibly helpful to productivity, to "print out or search the textual content of messages sent/received by my code" when developing and testing apps that communicate over they network. A text protocol has a huge advantage in that department--or rather, it has a very small advantage that is present in a very large number of development/testing tasks.
Also, readable plain text is key. XML dialects, or e.g. Thrift's JSON format are plain text, but are not terribly readable. This is subjective, to be sure, but makes a big difference in practice.
There are great reasons to use binary protocols, but any comparison that leaves out the benefits of text content that is readable "for free" without using per-protocol tools is lacking.