Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by neom on Wednesday January 01, 2025 @06:53PM
(#42570309)
She said it's around 60% there, but not helpful for her specifically as her area of research is on the families of the slave trade, and so in the document, that is actually the only thing that really matters to her. Names and places (she spends A LOT of time tracking transfer of slaves through states) - I guess I should be honest that I'm goalpost moving a bit from my original original post, it can work through some 18th english century text, but generally struggles where it matters, the details.
[ Parent
| on HN ]
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by whimsicalism on Wednesday January 01, 2025 @06:55PM
(#42570321)
i’m curious what the ground truth actually names in the doc are
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by neom on Wednesday January 01, 2025 @07:06PM
(#42570369)
Caroline Timmis is the boy mother, she is married to Henry Jenkins but was married previously, the boy is James Timmis.
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by CamperBob2 on Wednesday January 01, 2025 @07:20PM
(#42570446)
Looking more closely at the image, it seems that the strikeouts are what are confusing the model. It sees "Henry" and disregards "James" written above it. In a few places, the strikeouts almost look more like underlining.
Gotta be an insanely-challenging task for a program that wasn't even written with handwriting recognition in mind.
Other than the proper names, are any major details wrong?
Re:30% Drop In o1-Preview Accuracy When Putnam Problems Are ...
by neom on Wednesday January 01, 2025 @07:29PM
(#42570502)
Dr. Jang said: "I've not transcribed this work yet it's future work. I can send it once it's transcribed, however from skimming the various chatgpt texts you sent me, the people are generally wrong, their relationships are inconsistent between all the text you sent me, these issues are why I do not go to chatgpt to help me with my research. The document generally is indeed about requesting the courts oversee the apprenticeship work. Minors could work, however, when minors are used it needed to be overseen by the courts, this document is the court overseeing the work"
She provided this as one she just got done working with: https://s.h4x.club/z8u9xmv7 (John King Esq.)
I will also happily again admit a bit of goal post moving on my part. I was probably a little to harsh on it (maybe because I'm used to her and her history geeks talking about how they don't work well for their research).