Re:Tell HN: ChatGPT cut off date now Jan 2022
by dualityoftapirs on Tuesday September 19, 2023 @08:04AM
(#37568753)
They probably won't share how they did it, but there's been a lot of research over the past 6 months showing how you don't have to retrain the entire model to add in new sources. I know nothing about this stuff, but my limited understanding from blog posts is it's easier than anyone had thought to add in new data to a pre-existing model.
[ Parent
| on HN ]
Re:Tell HN: ChatGPT cut off date now Jan 2022
by skilled on Tuesday September 19, 2023 @08:05AM
(#37568781)
Do you by any chance have any of these blog posts available for my own reference? If not you maybe someone else does, I don't recall seeing it but it sounds interesting.
Re:Tell HN: ChatGPT cut off date now Jan 2022
by circuit10 on Tuesday September 19, 2023 @09:43AM
(#37569826)
I assume they mean fine tuning?
Re:Tell HN: ChatGPT cut off date now Jan 2022
by rafaelero on Tuesday September 19, 2023 @11:39AM
(#37571522)
I think there was a paper from Google showing that if you included 5% of your original dataset together with the new data during the finetuning then catastrophical forgetting didn't occur. Perhaps it's that simple.