They probably won't share how they did it, but there's been a lot of research over the past 6 months showing how you don't have to retrain the entire model to add in new sources. I know nothing about this stuff, but my limited understanding from blog posts is it's easier than anyone had thought to add in new data to a pre-existing model.

Do you by any chance have any of these blog posts available for my own reference? If not you maybe someone else does, I don't recall seeing it but it sounds interesting.

I assume they mean fine tuning?

https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)

https://platform.openai.com/docs/guides/fine-tuning

I think there was a paper from Google showing that if you included 5% of your original dataset together with the new data during the finetuning then catastrophical forgetting didn't occur. Perhaps it's that simple.

they are working on including information up to 2025. now THAT will be a big announcement.

Since gpt will be writing all the text, this is possible.

This is so true.

Chatgpt model is constantly training and it doesn’t require retraining the whole model.

Can you provide specifics?

Only that openai has said they are constantly changing the chatgpt model.

Right...