> obviously

This is not at all obvious to me. Symbolic reasoning feels quite different from picking the next word. Using physical demonstrations (or mental models of physical demonstrations) feels quite different from picking the next word.

Over the years I’ve come to believe that claims that something is “obvious” tell you more about the claimant’s state of mind than about the thing being claimed.

Symbolic reasoning isn't an atomic action. I can't speak for you or anyone else, but at least for me, symbolic reasoning involves either cached conclusions, or a fuck ton of talking to myself in my head - and that part is effectively LLM-like.

Which is why I'm still bewildered people expect LLMs to solve math and symbolic issues directly, when they're clearly (see e.g. "chain of thought") better treated as "inner voice" and used accordingly.

A lot of this kind of reasoning is very visual to me and involves no inner monologue of any kind - just visualizations flying around in my brain in complete silence. The translation into words happens later as a separate step. I wonder if this is not a universal experience.

Some people have no inner monolog, something that blew my mind.

When I work on problems I don't understand I'll monolog it internally. Now when I'm doing things I understand well I have some kind of internal shortcut language (my own token set I guess), that can sometimes make it difficult to explain to other people.

Can you describe how you would reason about say a paragraph of code--something which is very language based?

Depends on what the code is doing! Typically I “see” the underlying concepts or data structures interacting with each other. Imagine those block and arrow diagrams you would have in a textbook explaining how pointers work, except it’s completely automatic for me. My day to day involves a lot of low level systems stuff, so reading a page of code could spawn an image of how various event loops, memory allocations, etc. interact. Then to explain what I’m thinking of to a colleague is sort of like describing what I’m already “seeing” in my mind’s eye. Language to me is just an interface to this inner visual world.

Given the amount of people who report not having an internal monologue at all, I don’t think human logic is predicated on using words. They definitely can make complex logic easier, but it’s not a requirement.

The cabbage, wolf, goat problem is also an easy example of a problem that doesn't really need words to solve once you’ve conceptualized it. You can solve it by moving physical figures back and forth, either literally on a table or using the visual imagination part of your mind if you have one.

Not being aware of something is different from not having something. If said people still manage to carry on conversation, chances are good it's being generated somewhere internally. Everyone is self-aware to different extents and about different aspects of self.

> Given the amount of people who report not having an internal monologue at all,

What does this mean?

What is an internal monologue, exactly? Certainly I've heard to words a lot, but... what is it?

That thing. I don't do that.

It's thinking in words.

No, that's called aphantasia, it's orthogonal to not having an inner monologue (the "voice in your head"), and there are in fact people who effectively hallucinate on demand.

Which I suppose explains a lot of phrases that make little sense if they're only taken symbolically. Or why some people like long scenery descriptions in books - they can literally imagine it. Unfortunately, I'm aphantasic, so I can't.

Did you get a CT scan or MRI? Or self-diagnosed?

That ‘that’s what humans do too’ is also a repetitive statement. The truth lies somewhere in between, as always: humans and LLMs are similar, but in their current state LLMs do have a serious problem with reasoning tasks — even ones children find trivial.

What reasoning problem do children find trivial and LLM’s have serious problems with?

I can’t think of any off the top of my head.

I also tried to get GPT4 to craft such a problem and it was unable to: https://chat.openai.com/share/c1d5af4b-1d45-41ed-8f5a-746ea0...

Solving mazes. GTP4 got stuck at 7x7.

https://ekzhu.medium.com/gpt-4s-maze-navigation-a-deep-dive-...

My 4.5 yo wouldn't solve a 7x7 maze zero-shot either, not off-hand. Not even given multiple examples. Especially if getting confused, frustrated, and giving up is a valid move.

At which point and after how much training a kid becomes able to solve mazes like this? Also, given how one can pull a problem like this - any problem - out of their ass, describe it to GPT-4, and it has a good chance of solving it, that's quite amazing compared to children generally not being capable of this.

Arguably, but not absolutely.