News for hackers. Stuff that matters.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by thaumasiotes on Thursday January 02, 2025 @01:56AM (#42572357)
That's not acing the question. It's completely incorrect.

What do you think the singer in "Friends in Low Places" meant in the toast he gave after crashing his ex-girlfriend's wedding?

And I saw the surprise and the fear in his eyes

when I took his glass of champagne

and I toasted you, said "Honey, we may be through

but you'll never hear me complain"

Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by sdenton4 on Thursday January 02, 2025 @11:02AM (#42575564)
That requires knowing the song, beyond the words provided. Would you flunk an eighth grader for getting it wrong?
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by thaumasiotes on Thursday January 02, 2025 @12:47PM (#42576659)
Well, the rest of the song helps, in that it specifies that (1) the toast upset the wedding, and (2) the singer responded to that by insulting "you", which is presumably one or more of the bride, the groom, and the guests.

But I think specifying that the singer has crashed his ex-girlfriend's wedding is already enough that you deserve to fail if your answer is "he says he's not upset, so what he means is that he's not upset". It's not any kind of leap to guess that the bride's ex-boyfriend's toast might cause a scene at a wedding - that's why the bride's ex-boyfriends are never invited.

(The question has already provided every word of the toast that appears in the song.)

See also the sidethread comment by mikeruiz, noting that o1-pro reproduces the rest of the lyrics to The Victory, but gets the question wrong anyway.

Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by sdenton4 on Thursday January 02, 2025 @03:52PM (#42578735)
I was referring to the original query, of course, as any entity capable of reasoning could have figured out.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by thaumasiotes on Thursday January 02, 2025 @05:16PM (#42579544)
Hmm. Is there anything in my comment above that might address that point of view?
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by sdenton4 on Thursday January 02, 2025 @07:17PM (#42580600)
Nah, intermittent failures are apparently enough to provide evidence that an entire class of entities is incapable of reason. So I think we've figured this one out...
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by kbelder on Thursday January 02, 2025 @01:11PM (#42576981)
Sounds like you just proved ted_dunning isn't sentient.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ... by thaumasiotes on Thursday January 02, 2025 @05:13PM (#42579516)
Well, I proved that he's happy to express an opinion on whether an answer to a question is correct regardless of whether he knows anything about the question. I wouldn't trust advice from him or expect his work output to stand up to scrutiny.

Sentience isn't really a related concept.