Back in the 1990s, Ron Rivest came up with canonical S-expressions, which are fully capable of representing the same information represented by HTML, XML, ASN.1—but can be either human-readable or binary.
Here's a simple example: (p (* class (footer x-treme)) "This is a " (b "footer") "."). Very human-readable, very human-editable, and easily machine-readable, wouldn't you agree?
As a binary format, it would be (1:p(1:*5:class(6:footer7:x-treme))10:This is a (1:b6:footer)1:.). Still geek-manipulable, if necessary, and extraordinarily simple to parse. And it has the advantage that it is a distinguished encoding--any of the myriad human-readable encodings all reduce to the same canonical encoding, which has advantages for hashing.
Perl doesn't tend to have an explicit compilation step. You could write a HTML compiler to be as permissive as HTML parsers and you wouldn't have the same frustration you had trying to get started with perl.
If anything, HTTP/REST was much more complex. HTML was somewhat mystic yet allowed you to move things around easier, and actually design things vs just presenting them. And hyperlinks were really cool.
The web is big enough today that we can afford to make it more efficient since we have so many professionals that don't require the kind of implicit hand-holding provided by the original implementations. Back then everyone was a newbie and there were no web professionals.
I hate the idea that you would have to be a "web profesional" in order to get started today. We should embrace and keep the culture the web was built on.
All that said, I think peterwwillis is right: http/X.0, as long as it's well-simplified, is better in binary than it is in text. Ideally, there's a bijection between your text-mode and binary-mode (like a lens), where it's easy to parse (rely on your toolset to do the translation back and forth) and easy to put on the wire. Forth is a good example of how to do it sanely.
I would put those 11 and 12 year old down under "motivated independent learner"
If your concern is the a binary HTTP, too bad, it already here, its called HTTPS. It does seem to be holding anyone back.
If your concern is a binary HTML/compiler for it, I don't by that either. I was using a compiler when I was 8 years old and didn't have a problem with it conceptually.
Just because we have more 'professionals' today doesn't mean everyone is a professional.
Like with gaming, each game should be simple to start but deeper to master. That is what this layer is all about, lower down the OSI stack it is much more of a wall to beginners. Never lock out beginners as they can be better masters with time, don't hide the entrance to the labyrinth. Leave things as approachable, but with better professional experience, modifiable to perform better. We have all that now, a competing binary protocol will never see the innovation that a more simple one like HTTP will. Higher up the stack you can see how this was better for exchanging data in standard ways from old school binary blobs, to CSV files, to XML files, to JSON. Same reason REST services won over SOAP, simplify... There is a JSON binary format BSON in mongo and also MessagePack but guess which one is used more in service/exchanging data the textual one or the binary ones? The binary formats work well in certain situations where both endpoints are controlled by the same entity.
Binary and more locked down/optimized formats and messaging have their place but the start/base should always focus on simplicity over optimization.
The general rule in exchanging via standards is be liberal in what you accept, conservative in what you send. Being all binary all the time is a backwards step and is conservative on what is accepted, I also think it would lead to a host of difficult to debug problems just based on work I did with AS2/HTTP RFC implementation one of them being streaming and of course encoding/decoding which can fill hours of work if you can't visually see the content at some level.
There's something beautiful about being able to teach someone how to write a basic HTML document in 20 minutes. My mother can easily understand what HTML is doing, I doubt she'd understand a compiler.
Your argument doesn't really apply at all to HTTP, though. No one "got their start" peeking at HTTP requests. It's solely the domain of those working on infrastructure (for some definition of that word). Anyone with any idea of what HTTP really is (ie more than "that thing at the start of a URL") should have no problem using a tool to convert between a binary and text representation of the protocol. It's not like you can just magically pull HTTP requests out of the ether, you need a tool anyway. There's no reason why curl (for example) couldn't transform a request you write into an equivalent binary protocol, or why it couldn't do the inverse operation when it receives the response. It's utterly ridiculous to me that there is such inefficiency in HTTP just to make things slightly easier on the implementers of curl and wireshark.
That is not a viable option.
If you're aiming for an existing ecosystem, then sure, there's no reason not to use HTTP, assuming you make use of established libraries. But widespread use is HTTP's only real virtue; the protocol is considerably more difficult to implement correctly than it should be.
A minimal HTTP server that recognizes GET requests, finds the url, throws away everything else, runs the relevant code and returns an HTML document of the results is actually really easy to write. And easy to integrate into an existing event loop.
But even a minimal HTTP server, even one that ignores things like HEAD requests, is still going to be more complex than, say, receiving the URL raw, or even a URL wrapped in a simple structure like a netstring.
Actually implementing a full, correct HTTP server would be one or two orders of magnitude more complex that implementing a more modern protocol from scratch.
Just as an example I started playing around with HTML when I was about 12 (in 1998), it was easy and I got instant results. A year or so later I tried to learn Perl and quickly gave up because I couldn't get my first script to run. It was another year before I tried to "program" again and became hooked.
HTML and HTTP were successful because they were incredibly easy to understand. I wish we could keep it that way.