It almost belies a need for some sort of standardized testing as well. Self-reporting seems like a weak point that can be gamed.

To your point about luddites, that’s why I think it’s erroneous to use “as good as a human” as the metric. It will need to be much better than a human but those people reluctantly trust it enough to hand over control.