> The whole point of "fact checkers" is to be used by people who are not expert on the subject

A fact checker is a (usually tertiary, like an encyclopedia, providing references to primary and secondary sources) source that is distinguished by focussing on current, controversial claims. There is nothing more a "fact checker" can be. That's not a failing, that's an inherent limitation given the nature of facts.