Essentially, for "real-life work scenarios", the performance is not that great comparing to gpt-3.5 with the exception of Claude 3.7 and GPT-4.5. Bear in mind, the question was not "challenging" in a "genius thinking required" way.
That being said, I use LLM regularly for discovery. They do waste some of your time because of hallucinations but you can call their bullshit most of the time and if not, it is still faster than Googling.
Re:IBM's CEO doesn't think AI will replace programmers anyti...
by yahoozoo on Monday March 24, 2025 @08:26AM
(#43460218)
You steer ChatGPT 3.5 model to find the handlers HashMap issue that it didn’t find initially, but you dismiss the other models for not immediately recognizing it?
Essentially, for "real-life work scenarios", the performance is not that great comparing to gpt-3.5 with the exception of Claude 3.7 and GPT-4.5. Bear in mind, the question was not "challenging" in a "genius thinking required" way.
That being said, I use LLM regularly for discovery. They do waste some of your time because of hallucinations but you can call their bullshit most of the time and if not, it is still faster than Googling.