They are "capable", but only by chance. If you generate millions of random sentences in a human language then some of them would express some "useful new solutions". LLMs would have a higher chance of this happening as they generate sentences that are statistically more probable and therefore more likely to represent a coherent thought, but this would still be a chance.