Its a broken window, if you don't fix it people have permission to just have tests that fail, they get used to seeing a failure that 2 failures isn't an issue and pretty soon real problems that would be caught by the tests aren't fixed anymore.

A nice thought, but pointless. Failing tests do not matter to the PHP release managers, they ship it anyway.

http://redd.it/qeq7k#c3z0vva

That's nonsense, PHP releases ship with no failing tests. That's from three years ago, and anyway gcov.php.net is not used for CI these days (IIRC it's an abandoned old box, its results are inaccurate), Travis is. The PHP-7.0 and PHP-5.6 branches are currently green on Travis.

That also reminds me back when I did x.400 mail for a living (at british telecom) Sprint had a implementation that completely contradicted the ITU standard - Sprints response was well just put a fix in for sprint's mail systems.

ICL also though that even if the standard said such and such index MUST start at 1 that they woudl start it at 0 - cue some nice divide by zero errors - Not surprised we don't have a UK Mainframe company any more.

Similarly, back when I used to maintain Stackless Python and do merges in from mainline Python, my benchmark was to only have the same set of failing tests post-merge for the same merged revision.

More often than not I would have failing tests in official Python release version tags, which became par for the course.

Sounds like a good idea to me!