Re:Why TLS 1.3 isn't in browsers yet
by gsnedders on Wednesday December 27, 2017 @08:01PM
(#16019472)
The WHATWG and the W3C have been moving in a very different direction, treating testing as a major part of spec development, because the goal is to get implementations of the specifications, and how do you determine that if not by testing.
https://blog.whatwg.org/improving-interoperability is a post from the WHATWG side about this, and the outcome isn't just the direct "it's easier to write code against tests" but also "it's easier to notice when the spec changes" (because you now get a failing test).
The vast majority of RFCs don't have conformance testing suites. IETF focuses on interoperability testing, which demonstrates that independent implementations work together for the major use cases, it doesn't test all the edge cases (such as future version negotiation) that a good conformance test would.
Creating and maintaining a conformance test is an order of magnitude more work than creating or maintaining a specification.
Someone has to be willing to expend the effort, and that usually means it has to make economic sense, and it rarely does.
And even when it does, in order to gain funding it often has to be licensed on commercial terms, which puts it out of reach of a lot of open source projects and smaller commercial implementors. (Even many large companies wouldn't pay for a test suite unless customers start putting it in RFPs.)
NIST used to maintain a whole bunch of free conformance test suites (CGM, COBOL, FORTRAN, PHIGS, POSIX, SQL), but the US government decided to stop paying for that.