News for hackers. Stuff that matters.
Re:Re-Evaluating GPT-4's Bar Exam Performance by hyperman1 on Monday June 03, 2024 @01:42PM (#40565171)
Not yet, because the reliability isn't there. You still need to validate everything it does.

E.g. I had it autocompleting a set of 20 variable#s today Something like output.blah=tostring(input[blah]). The kind of work you give to a regex.

In the middle of the list, it decides to go output.blah=some long weitd piece of code, completely unexpected and syntactically invalid.

I am still in my AI evaluation phase, and sometimes I am impressed with what it does. But just as possible is an unexpected total failure. As long is it does that, I can't trust it.

There are no comments on this comment yet.