If you want to scrape websites or show them in a browser, then you have to follow the needs of makers of these sites, because you need them and they don't need you. If you want to go to the right and they want to go to the left, you either follow them or go alone and become unable to scrape or browse their content. If there's a feature that they want to use that makes your parsing more complicated, tough luck, that feature is going in as long as somebody (e.g. major browser vendors) will agree to make it work.
XHTML would have worked the same way – after a few years, you can deprecate the old parsers.
There are people who are now dead who have pages on the internet. These pages will never be updated.
I guess I don't get your huge bone to pick with Selenium.
Try running hundreds of tests at the same time, to actually get fast results when running a full test suite.
Right now running a small testsuite takes here over 2 hours, 99% of the time is spent in the browser processes.
And that’s not nearly close to 100% test coverage.
> not worth getting rid of HTML5 to abandon it
You don’t have to – XHTML isn’t the only strict spec out there, AMP is another, React also enforces strict syntax in JSX templates. AMP fails to render anything if there’s even a single mistake, React fails at build time.
You can get an HTML parser off the shelf. A browser vendor (Mozilla) even funded one for non-browser purposes before adopting it for Firefox.
It didn't take "thousands of Google developers years" to teach a computer English spelling rules. Indeed, even a programming class assignment could do that: you can cover most cases by just looking the pronunciations up in a dictionary. (The number of quirks and inconsistencies makes English spelling quite hard for humans to memorize, but computers are rather good at lookup tables.)
Even if you consider the actual Vocaloid software, which was developed by a team at a large corporation, there are two factors differentiating it from English TTS that make the latter much harder:
1. Japanese has much simpler phonetics than English, with a smaller set of phonemes and (somewhat oversimplifying) only using open syllables. So it's easier to consume and produce, for both computers and humans, but at the cost of being a less efficient encoding: Japanese tends to require a lot more syllables than English to express the same concept, and there are a lot of homophones.
2. Vocaloid sounds robotic. It's gotten a bit less so over time, but it still doesn't come close to passing as human. If you're okay with robotic, English TTS software has existed for a long time, starting many decades before Google was founded. The hard part, the part that requires neural networks and massive computational power and Google and still has yet to be perfected, is making it sound human.
By the way, although vocaloid software would be given phonetic input, normal Japanese writing uses kanji (i.e. Chinese characters), most of which have multiple unrelated possible pronunciations. Determining which pronunciation applies to each character in a given piece of text is nontrivial, and sometimes even depends on context or meaning.
But if you actually try to write a new browser from scratch, or a tool to scrape websites, you’ll learn to love XHTML, and hate HTML5.
The same that applies with human languages applies here as well. Writing a vocaloid for japanese is a high school programming class project. Writing a TTS for english takes thousands of Google developers years. Writing an XHTML1.1 parser and renderer takes a month. Writing an HTML5 browser takes thousands of developers years.
The WHATWG specs prevent the web from ever evolving further – we’re stuck with opaque websites and no way to build new technology on top of it. Building a crawler is a task of years, and semantic data tags are impossible to parse, because no one uses them right.
The only reason we can automate any parsing of websites is because either browser vendors spent billions on crawlers for their search engines, or if we run a full browser, or if we use the opengraph tags that Facebook forced on websites.
With XHTML1.1, Chrome and Firefox headless would never have been necessary. Imagine how much time and computational cost you could save.