> LLMs are completely unpredictably 'bad' - sometimes a task that a high schooler can solve sends an LLM into a spin, sometimes an LLM solves a task that would require a PhD in the field.

That doesn’t mean they are unpredictably bad, it means assuming that they behave like humans is a bad way of predicting their behaviour. “Unlike humans” and “unpredictable” are not the same thing. If you spend time working with them, you get a better sense of what they are good and bad at and get better at predicting their behaviour.