This is literally how anything every has started :)
> The goal seems to be to be able to build any kind of application as a web app
Been doing this for several years... it's really unpleasant... and I love programming.
There are numerous "HTML page simplifiers" (most based on Readability's engine AFAIU), which might shim behaviour and compatibility for legacy pages.
And content itself is text, not code. Slavish backwards compatibility is not a strict requirement.
I think that's really the key to this that makes the "worse is better" argument miss the mark here. HTML succeeded because anyone could open up a text editor, learn a few simple rules, and have a web page in short order. It's fantastically more complicated now, but unlikely to be supplanted because we have two and a half decades of HTML+CSS+JS out in the wild. People work hard on cross-browser compatibility because it's not going away, not because it's fun.
And all that fragile, brittle content out there will eventually break. The question is when compatability is lost, and in the name of what.
Keep in mind that I'm specifically targeting text and textually-oriented document content. The modern Web can be considered generally as having four principle modes, three ofwhich I'd treat separately: documents, as described, commerce (probably hived into a dedicated application), media (likewise), and apps (which want a VM engine, e.g., Chromium).
A surprisingly large set of apps, and certainly many significant ones, are principally document-and-discussion engines, for which lack of an intrinsic model within the document markup and client presentation is the raison d'etre of those apps. Either having a paired discussion platform, or integrating discussion into the browser itself, would address much of this.
Other content elements which have become significant online include both advertising and DRM. These have been mistakes.
You lost me there. If you want any kind of adoption, you need to be backwards-compatible as much as possible. Otherwise you're just building a toy for geeks to play with, and you end up only attracting "spec perfectionists" to work on it (that is, people who care so much about the spec/implementation being beautiful and elegant that they never successfully ship something people can use).
> HTML would have to be written correctly (eg. balanced tags)
This is a common misconception, unless you're talking about XHTML (which was mostly a failure adoption-wise). HTML is a variant of SGML, which does not require balanced tags (though you can specify that certain tags must be balanced, of course). Certainly you'd prefer to enforce them in some cases where it makes sense (like <em>), but things like <br> do not need a closing tag (and do not need to be expressed as <br/>).
Anyway, I think the overall issue with the web today is that people want it to be a complete application development platform + document layout system. The goal seems to be to be able to build any kind of application as a web app, and allow them to do anything a native app could do (though hopefully with better security). Not saying this is a good or bad thing, but if that's the goal, complexity is inevitable.