News for hackers. Stuff that matters.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by jstummbillig on Wednesday January 01, 2025 @12:05PM (#42567324)
FYI: If you do that without a subscrpition, you currently (most likely) get a response generated through 4o-mini — which is not any of their reasoning models (o1, o1-mini or previously o1-preview) of the branch discussed in the linked paper.

Notably, it's not even necessarily 4o, their premiere "non-reasoning"-model, but likely the cheaper variant: With a free account the model it claims to be using is "4o auto", which is not a model but apparently an attempt to automatically decide models for you to be more cost effective.

Without a ChatGPT subscription you can't select a specific model anymore, not even rate limited, as was previously possible.

Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by jsheard on Wednesday January 01, 2025 @12:24PM (#42567460)
There doesn't seem to be a way to choose a model up-front with a free account, but after you make a query you can click on the "regenerate" button and select whether to try again with "auto", 4o, or 4o-mini. At least until you use 4o too many times and get rate limited.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by evertedsphere on Wednesday January 01, 2025 @02:12PM (#42568390)
you can select the model in the header bar when you start a chat: the name of the currently selected model can be clicked to reveal a dropdown
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by jstummbillig on Wednesday January 01, 2025 @02:42PM (#42568659)
Are you on the free version? Because for me it did not show there, only on the paid one.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by jsheard on Wednesday January 01, 2025 @02:19PM (#42568463)
That option isn't there for me, maybe it's an A/B test thing.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by jstummbillig on Wednesday January 01, 2025 @01:06PM (#42567801)
Ah, interesting!