Re:Generative Language Modeling for Automated Theorem Proving
by riku_iki on Wednesday September 09, 2020 @12:34PM
(#24422767)
> but the penalty is that they need lots of training data and compute.
I think it depends on how noisy and unstructured data is (e.g. human language), transformer will likely approximate few well correlated datapoints just fine.
Re:Generative Language Modeling for Automated Theorem Proving
by nestorD on Wednesday September 09, 2020 @02:44PM
(#24424290)
Here the train set was only 1000 proofs (with an average of 90 steps per proof) so it is decidely a small dataset (but highly structured and noiseless).
I think it depends on how noisy and unstructured data is (e.g. human language), transformer will likely approximate few well correlated datapoints just fine.