They still can't. But very smart humans constructed ways to use the monkeys with typewriters (with a statistical advantage) to find correct answers to problems where they already knew how to verify the answer.

You say it like it's not an achievement.

No, he says it's a different category of problem, and ability to solve one doesn't carry over to the other and it doesn't imply intelligent understanding.

Which problems are not amenable to that approach?

Problems where verification of the proposed solution has a high cost or high risk.

Most of them.

Unfortunately, most problems aren't.

You need a problem where you both know what the solution looks like or can otherwise very quickly and efficiently determine that a solution is correct, but at the same time can't work out a correct solution with a similar amount of effort/time/cost as it took to determine how to verify a solution.

Most problems don't match that criteria. You usually either have a problem with a known method of solving, or you have a problem with no clear way of verifying the solution besides the act of finding the solution itself which would involve in some way proving it is correct, or you have a problem where verifying a solution takes a very long time or has a high cost or even can't be done more than once, so you need to try to determine the best solution without being able to actually test or verify.

Basically all problems just don't fit the "hard to solve but easy to verify" criteria to a degree that makes llms a good fit. On the other hand, there are so many problems that even a tiny fraction is a relatively large number.

You understand that those are different kinds of "truth", right?

Right, which was true at the time. So hundreds of billions of dollars have been poured into making LLMs better at these tasks via pretraining, RL, RLHF, post training, etc. again all with something verifiable in the loop. In order to improve the thing in the loop, the loop itself needs to be verifiable.

There have only been a few thousand wars, and they’re all different and all different in the world in which they occurred. The dimensionality is absurd, which is not a problem for LLMs if there’s enough data, but in this case there isn’t.

> they don't understand the notion of truth so they can't do verifiable work like math proofs.

Nobody that understands automated proof checking was claiming that.

> they don't understand the notion of truth so they can't do verifiable work like math proofs."

a) No one ever said that.

b) Your comment shows a lack of understanding of the notion of truth.