News for hackers. Stuff that matters.
Re:Re-Evaluating GPT-4's Bar Exam Performance by munchler on Sunday June 02, 2024 @05:53PM (#40557446)
> On any topic that I understand well, LLM output is garbage: it requires more energy to fix it than to solve the original problem to begin with.

That's probably true, which is why human most knowledge workers aren't going away any time soon.

That said, I have better luck with a different approach: I use LLM's to learn things that I don't already understand well. This forces me to actively understand and validate the output, rather than consume it passively. With an LLM, I can easily ask questions, drill down, and try different ideas, like I'm working with a tutor. I find this to be much more effective than traditional learning techniques alone (e.g. textbooks, videos, blog posts, etc.).

Re:Re-Evaluating GPT-4's Bar Exam Performance by LamaOfRuin on Sunday June 02, 2024 @07:15PM (#40557883)
I would classify all of those as "non-traditional" learning techniques, unless you actually mean using a textbook while taking a class with a human teacher.

Well written textbooks are consumable on their own for some people, but most are not written for that.

Re:Re-Evaluating GPT-4's Bar Exam Performance by djmips on Sunday June 02, 2024 @07:55PM (#40558097)
That's a good observation about textbooks and helps explain why I had difficulties trying to teach myself topics from a textbook alone!
Re:Re-Evaluating GPT-4's Bar Exam Performance by ghaff on Monday June 03, 2024 @10:19AM (#40562871)
A lot just aren't very good but they also tend to make assumptions about prior knowledge in line with what would be typical prerequisites for a class and some degree of guidance.
Re:Re-Evaluating GPT-4's Bar Exam Performance by erhaetherth on Monday June 03, 2024 @02:53AM (#40560019)
Might be better to think of the LLM as the student, and you're an imposter tutor. You're trying to assess the kid's knowledge without knowing the material yourself, but the kid is likely to lie when he doesn't know something to try to impress you, hoping that you don't know your stuff either. So you just have to keep probing him with more questions to suss out if he's at least consistent.
Re:Re-Evaluating GPT-4's Bar Exam Performance by ptx on Monday June 03, 2024 @10:38AM (#40563061)
I've had teachers who didn't understand the subject they were teaching. It's not a good experience and replicating that seems like a terrible idea.
Re:Re-Evaluating GPT-4's Bar Exam Performance by burnished on Monday June 03, 2024 @11:31AM (#40563685)
A key advantage is that LLMs dont have emotional states that need to be managed.