OpenAI employee: GPT-4.5 rumor was a hallucination

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by atleastoptimal on Sunday December 17, 2023 @05:35PM (#38677138)

My theory is that they're using a fine-tuned version of GPT-4-turbo in ChatGPT to fix the issue with everyone saying it was lazy. They fine-tuned it using responses from GPT-4.5-turbo, which they have internally. This synthetic data was probably based on a suite of questions which included self-identification, leading to it releasing its model name in the training data, and the new ChatGPT less lazy model is adopting the name via osmotic data leakage.

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by arthurcolle on Sunday December 17, 2023 @05:55PM (#38677259)

Great theory, this makes a lot of sense. It looks like they already RLHF'ed (I keep using this as a verb, but maybe its really just instruction tuning?) this behavior out of ChatGPT! https://news.ycombinator.com/item?id=38677025 Really fast, I wonder how this works. Anyone have any ideas / knowledge on how they deploy little incremental fixes to exploited jailbreaks, etc?)

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by krisoft on Sunday December 17, 2023 @06:02PM (#38677299)

> Anyone have any ideas / knowledge on how they deploy little incremental fixes to exploited jailbreaks, etc?

LoRa[1] would be my guess.

For detailed explanation I recommend the paper. But the short explanation is that it is a trick which lets you train a smaller, lower dimensional model which when you add to the original model it gets you the result you want.

1: https://arxiv.org/abs/2106.09685

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by arthurcolle on Sunday December 17, 2023 @07:27PM (#38677887)

is this another expert? can they hot swap in experts and remove others?

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by azeirah on Sunday December 17, 2023 @05:58PM (#38677277)

Not sure about this scenario precisely but I believe when it comes to filtering certain responses they have a separate model running in a similar manner to what meta recently released. I believe it's called llama purple?

https://ai.meta.com/llama/purple-llama/

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by idopmstuff on Sunday December 17, 2023 @07:52PM (#38678038)

Or as they updated its knowledge cutoff, it picked up references to gpt-4.5-turbo. They wouldn't have to be internal - plenty of people have speculated about the possible release of a model with that specific name.

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by atleastoptimal on Sunday December 17, 2023 @09:00PM (#38678472)

Why would it be deceived that it is a model that hasn't been released yet though? There would be more explicit references to gpt-4-turbo in the general internet training data

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by toastmaster11 on Sunday December 17, 2023 @07:01PM (#38677708)

But why would it be gpt-4.5-turbo, as opposed to just gpt-4.5? Are they just going to abandon past naming conventions?

If they already had 4.5 and then did something else to make it “turbo” why not release 4.5 on the coattails of Gemini?

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by zamadatix on Sunday December 17, 2023 @08:45PM (#38678370)

I don't think there would be pressure to rush a new model out immediately when Gemini Ultra (the one which is supposed to be better than the current product) doesn't even have a release date yet.

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by toastmaster11 on Monday December 18, 2023 @02:10AM (#38679965)

I’m asking, why is it 4.5-turbo when 4.5 isn’t released yet?

[ Parent | on HN ]

Re:OpenAI employee: GPT-4.5 rumor was a hallucination by jug on Sunday December 17, 2023 @06:00PM (#38677283)

Yes I still think it’s involved because hallucinations don’t use to be this coherent across the user base.

[ Parent | on HN ]