News for hackers. Stuff that matters.

A comment on GPT-4.5 or GPT-5 being tested on LMSYS?.

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by skissane on Monday April 29, 2024 @08:26PM (#40205935)
gpt2-chatbot is not the only "mystery model" on LMSYS. Another is "deluxe-chat".

When asked about it in October last year, LMSYS replied [0] "It is an experiment we are running currently. More details will be revealed later"

One distinguishing feature of "deluxe-chat": although it gives high quality answers, it is very slow, so slow that the arena displays a warning whenever it is chosen as one of the competitors

[0] https://github.com/lm-sys/FastChat/issues/2527

[ Parent | on HN ]
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by concurrentsquar on Monday April 29, 2024 @11:52PM (#40207063)
> One distinguishing feature of "deluxe-chat": although it gives high quality answers, it is very slow, so slow that the arena displays a warning whenever it is chosen as one of the competitors

Beam search or weird attention/non-transformer architecture?

[ Parent | on HN ]
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by pennomi on Tuesday April 30, 2024 @09:28AM (#40210771)
It has a room full of scientists typing out the answers by hand.
[ Parent | on HN ]
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by Mkengine on Tuesday April 30, 2024 @01:42PM (#40213720)
I looked 3 times through the list, I can't find a "deluxe-chat" there, how do I select it?
[ Parent | on HN ]
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by skissane on Tuesday April 30, 2024 @05:29PM (#40216594)
It has only ever been available in battle mode. If you do the battle enough you will probably get it eventually. I believe it is still there but doesn’t come up often, it used to be more frequent. (Battle mode is not uniformly random, some models are weighted to compete more often than others.)
[ Parent | on HN ]