Re:OpenAI’s policies hinder reproducible research on languag...
by loveparade on Thursday March 23, 2023 @12:19AM
(#35270496)
Given how small the time window between the successive releases was it's extremely unlikely that there were any big changes to the model. Most likely it's just better preprocessed training data, more training data, trained for longer, performance optimizations for attention, or a few changes to layer sizes.
[ Parent
| on HN ]
Re:OpenAI’s policies hinder reproducible research on languag...
by mach1ne on Thursday March 23, 2023 @02:25AM
(#35271087)
Your timeline is wrong, GPT-4 finished training already in August.
Re:OpenAI’s policies hinder reproducible research on languag...
by Veen on Thursday March 23, 2023 @03:16AM
(#35271439)
They didn’t stop release GTP-4 immediately after it was trained and then move on to training GPT-5. They had 4 for almost 6 months before it was released. 5 was certainly well underway long before we’d heard of 4.