News for hackers. Stuff that matters.

alexlyzhov

Re:Generative Language Modeling for Automated Theorem Proving on Thursday September 10, 2020 @03:31PM (#24435692)
OK it looks like this http://www.tptp.org/CASC/ is the standard benchmark for theorem provers. But I don't know the difference between evaluation schemes for Metamath and for CASC and I don't know why DL people evaluate on Metamath instead.

Isn't GPT-f SOTA on Metamath test though?

Re:Generative Language Modeling for Automated Theorem Proving on Thursday September 10, 2020 @03:19PM (#24435598)
> Besides if I understand correctly, Metamath, was used as the "ground truth" for evaluation and testing of the trained language model and the language model didn't get anywhere near 100% performance, meaning it's far behind the ability of Metamath in automated theorem proving.

I'm new to this but wiki says Metamath refers to language + proof checker. It doesn't refer do a method of proof search right? Isn't it just a database of proofs? I'm confused.

Can you please shine light on traditional approaches that are better than DL approaches at proof search? And on how the setting they are evaluated in is different from the evaluation setting in this paper?