News for hackers. Stuff that matters.
Re:This working group has failed by lazyjones on Tuesday January 07, 2014 @05:05AM (#7016491)
Specs designed by committee usually fail when they produce just the "paper" and no reference implementation. Look at W3C, which consistently failed to develop their own reference browser. The same seems true for TLS, actual implementations are too complicated because the committee paid no attention to this. A positive example is MP3, where everyone just copied the reference codec in the beginning.
Re:This working group has failed by octo_t on Tuesday January 07, 2014 @06:06AM (#7016629)
A good example here would be C++. Look at how long it took for C++11 to be implemented, but now at least two compilers are feature complete (gcc + clang) and now they're on schedule to be feature complete for C++1y[1], with implementation + specification in sync.

[1] http://clang.llvm.org/cxx_status.html#cxx14

Re:This working group has failed by MichaelGG on Tuesday January 07, 2014 @06:37AM (#7016733)
It also helps that C++ doesn't have nearly the number of implementers in position to screw stuff up. Unlike any number of webservers, SSL libraries, load-balancers, firewalls, proxies, etc.

How many C++11 compilers are there? This support matrix[1] seems to indicate there are 2 compilers in the lead, with 5 providing fairly decent coverage. How popular is XLC++?

And anyways, adding another compiler is something you choose to do, and usually run a bit of tests for, right? Unlike networking protocols where it's invisible to you that someone has some idiotic inspection hardware that segfaults on certain packets and drops your connection.

Edit: Also, even if two compilers are feature complete, it doesn't mean they implement the features in perfectly compatible ways, right?

1: https://wiki.apache.org/stdcxx/C%2B%2B0xCompilerSupport

Re:This working group has failed by Sharlin on Tuesday January 07, 2014 @10:05AM (#7017618)
And C++11 itself was implemented awfully quickly compared to C++03, exactly because this time around the Committee was much stricter about having reference (or at the very least prototype) implementations available before accepting anything into the Standard. As you mentioned, C++14 actually seems to be proceeding like it optimally should, so the Committee has certainly learned its lessons regarding this.
Re:This working group has failed by rjzzleep on Tuesday January 07, 2014 @12:48PM (#7018600)
websockets at the very least had a reference implementation. but it was more along the lines of "it works for me, i tested it on my local machine"

it took ages long discussions to get the point across. "yes, but on your local machine you don't have a reverse proxy like every other normal web application stack". it's going to break if we do it this way and they just kept responding.

"but it works here"

what about the oauth2 spectactle?

http://hueniverse.com/2012/07/oauth-2-0-and-the-road-to-hell...

Re:This working group has failed by gsnedders on Tuesday January 07, 2014 @02:11PM (#7019158)
At the W3C, no spec can reach REC without having two interoperable implementations.

Also note everyone copying the reference implementation isn't good either — there's no guarantee the spec actually defines enough to recreate the reference implementation then!

Re:This working group has failed by lazyjones on Tuesday January 07, 2014 @04:53PM (#7020257)
> At the W3C, no spec can reach REC without having two interoperable implementations.

Wishful thinking? Where are these reference implementations documented for HTML 2.0, 3.2, 4.01?

> Also note everyone copying the reference implementation isn't good either — there's no guarantee the spec actually defines enough to recreate the reference implementation then!

If the spec is lacking, whatever the reference implementation does completes the spec. There is no value in having a pure paper spec that someone may be able to implement a correct piece of software with over having a correct reference implementation with full source code.

Re:This working group has failed by gsnedders on Tuesday January 07, 2014 @06:56PM (#7020853)
> Wishful thinking? Where are these reference implementations documented for HTML 2.0, 3.2, 4.01?

There is no requirement for them to be "reference implementations". CSS 2.1, for example, used various web browsers as proof of interoperability (where every test in the test suite was shown to have at least two passing implementations). The "interoperable implementations" clause was introduced as a recommendation in 1999, 2001 saw it required every feature in the spec must have been implemented, preferably with two interoperable implementations, and clarified as RFC2119 "should" in 2003 — HTML 4 was published in 1998, prior the W3C having a formal process document, and HTML 4.01 was likely exempted as a minor revision in 1999.

> If the spec is lacking, whatever the reference implementation does completes the spec. There is no value in having a pure paper spec that someone may be able to implement a correct piece of software with over having a correct reference implementation with full source code.

Ideally the spec shouldn't be lacking. If one needs a reference implementation, the spec has failed to be a complete definition, and if the spec is not a complete definition, what is the point of it? You may as well just have a high-level overview in the documentation of the reference implementation!

Re:This working group has failed by st0neage on Tuesday January 07, 2014 @07:43PM (#7021016)
And you are citing this as an example of how to do it right? Why?
Re:This working group has failed by gsnedders on Wednesday January 08, 2014 @12:29PM (#7024918)
> And you are citing this as an example of how to do it right? Why?

Because reference implementations are a bad form of spec. (Note I'm only arguing against normative reference implementations — purely informative ones aren't really a "reference" any more, they're just another implementation, unique in no way except perhaps being the first.)

Why? Because:

- If any disagreements between spec and reference implementation are settled by the reference implementation being right, then there is no motivation to write anything except, "I like unicons". Because, after all, the reference implementation says how it is actually done.

- Similarly, if the reference implementation is treated as being right in case of contradiction, if it has a null-pointer dereference bug leading to a SIGSEGV (per the de-facto standard!), every third-party implementation must SIGSEGV in the same case — because, after all, that's what the spec says you must do!

- Reverse-engineering an implementation is often far harder than understanding a spec, making an independent implementation more difficult to create (if you just want the reference implementation used everywhere, why bother standardizing it?), as one has to get the abstract model out of the reference implementation (which may be implicit and not stated anywhere). This is especially relevant if one wishes to make an implementation using a different model to the reference (e.g., a parallel implementation of something with a sequential reference implementation).

- If everyone uses the reference implementation, the spec (which, in that case, is the reference implementation) is less likely to have so many eye-balls looking over it (looking for inconsistencies, unintended behaviours, ambiguities, and outright bugs) than if it was implemented by multiple, independent people/teams.

Having a spec developed with multiple independent (non-normative) implementations:

- Leads to a clearer spec, having had to be read unambiguously by multiple implementers;

- Leads to greater peer review of implementations, as with a reference implementation it can easily become the only implementation of the spec, thus having to only interoperate with itself, whereas requiring multiple independent implementations requires that they must be able to interoperate; and

- Leads to higher quality implementations, as they are required to match the spec (an external, normative, spec places constraints on the implementations, and therefore leads to better review of the implementations as well as the spec!).

One certainly doesn't want a spec to developed in isolation from any implementation — but one doesn't need a normative reference implementation to garner the advantages of being developed in tandem with one. Multiple independent implementations are needed to ensure multiple people can understand the spec without having to resort to guessing the author's intention (likely differently!).

Ambiguity tends not to be a major issue in a well-written spec, and while certainly a formal spec (such as a reference implementation) does define all cases, it will also define formally all bugs that the author wrote because they are a fallible human. Bugs in specs will occur no matter how formally they are stated and they are a far bigger issue than ambiguity. The important part is to get as much peer review as possible of the spec to maximize the chance the bugs are found — and if you have multiple implementers all reading the spec while implementing it, they are more likely to catch the bugs than someone just taking the reference implementation and using that.

Re:This working group has failed by colanderman on Tuesday January 07, 2014 @11:26AM (#7018095)
Look at W3C, which consistently failed to develop their own reference browser.

Does http://www.w3.org/Amaya/ not count?

Re:This working group has failed by robin_reala on Tuesday January 07, 2014 @01:28PM (#7018857)
It exists, but it’s hardly a reference for the current state of the specifications.