Re:NeurIPS 2020 “English to Bash” Competition Goes Live
by speedgoose on Tuesday July 21, 2020 @03:42PM
(#23910173)
Isn't the whole point of GPT-3 to have tons of data and hope for the best by training it with a huge amount of time and energy ?
Re:NeurIPS 2020 “English to Bash” Competition Goes Live
by setr on Tuesday July 21, 2020 @03:54PM
(#23910308)
For training -- using the trained algorithm is cheap. It's a one-time cost (well, one-time in the final version)
Re:NeurIPS 2020 “English to Bash” Competition Goes Live
by DSingularity on Tuesday July 21, 2020 @04:03PM
(#23910396)
You do realize that reducing costs means dropping parameters. So what you are suggestion is reversing the very technique that enables this result. Color me doubtful.
The second version can require three orders of magnitude less hardware. Premature optimization is the root of all evil.