Re:I wrote JustHTML using coding agents
by Aloisius on Friday December 19, 2025 @03:01PM
(#46330189)
run_tests.py does not appear to be checking the number of errors or the errors themselves for the tokenizer, encoding or serializer tests from html5lib-tests - which represent the majority of tests.
There's also something off about your benchmark comparison. If one runs pytest on html5lib, which uses html5lib-test plus its own unit tests and does check if errors match exactly, the pass rate appears to be much higher than 86%:
$ uv run pytest -v
17500 passed, 15885 skipped, 683 xfailed,
These numbers are inflated because html5lib-tests/tree-construction tests are run multiple times in different configurations. Many of the expected failures appear to be script tests similar to the ones JustHTML skips.
Re:I wrote JustHTML using coding agents
by EmilStenstrom on Friday December 19, 2025 @05:38PM
(#46331789)
Excellent feedback. I'll have a look at the running of html5lib tests again.
Re:I wrote JustHTML using coding agents
by EmilStenstrom on Sunday December 21, 2025 @07:45AM
(#46344448)
I've checked the numbers for html5lib, and they are correct. They are skipping a load of tests for many different reasons, one being that namespacing of svg/math fragments are not implemented. The 88% number listed is correct.
That said, the example you are pulling our out does not match that either. I'll make sure to fix this bug and other like it! https://github.com/EmilStenstrom/justhtml/issues/20