I got

"Here is a random string of 32 characters:

a8Jk5pYr0Dm9Nc1Vz8Qf2Bt6Hg3Lw4Uo"

Wouldn’t be surprised if it’s good at 32 because it’s a power of 2.

I’ve tested this with a wide variety of number inputs and it’s performance is highly variable. Error also increases linearly with strong length.

If you asked most people to perform that task, they literally wouldn't have a clue what you'd just asked them to do.

They have a specific device to do that now. I have tried to say "write a random sentence with 6 words and 2 numbers" and it completely fails, but it can do the straightforward "write a random [x] of length [y]."

a 4 year old would fail at this task.

does a 4 year old have intelligence?

Yup. I think this is the best point of comparison - a 4-6 year old kid. Specifically, one that hasn't gone to school yet. The difference between a typical 6-year old and a typical adult is in big part that the latter spent 10+ years being systematically fine-tuned.

Logic, arithmetics, algebra, precisely following steps of an algorithm - those are not skills one "kinda" just "gets" at some point, they're trained by deliberate practice, by solving lots and lots of problems specifically constructed to exercise those skills.

Point being, get GPT-4 through school, and then compare with adult performance on math-adjacent tasks. Or at least give it a chance by prompting it to solve it step-by-step as a problem, so it can search closer to the slice of latent space that encodes for relevant examples of similar problems and methods of solving them.

I started seriously using computers at 2.5, and I started writing and recording songs with a tape recorder at 3, won a local award for one song, and playing chess at 4. I know plenty of people with similar experiences. If you nurture kids and don't treat them like they're stupid, they can do some quite impressive things.

Anecdote: admittedly, I'm autistic as are the people I know, so maybe that's not a good sample. I struggle with a lot of basic shit even as an adult. Oh god, I empathize with the hypothetical GPT5.

I did not say GPT4 did not have intelligence. I gave an example of a task it fails that is easy for most humans.