I have yet to see an LLM that is cooperative. The magic of collaborating with someone is that we can both understand the problem and reason about it.

The current degree of LLM intelligence is not compelling for a social creature like me.

Is it even allowed to ask questions??

Edit: my sience fiction joke in the 90s was AI though bots chatting in irc channels. They could seemlesly integrate human intelligence that way.

Surprised to read that.

I use them as a cooperative partner by default.

Also: quite a few people have had instances work with other instances, sometimes of the same model and sometimes of other models.

Cooperation is more than an i/o loop. Layering and pooling models is nice though.

Perhaps "conceptualization" is the indicator here.

Perhaps I'm up too late, but I can't think what else is there to cooperation besides two or more agents doing things in alignment with some goal? (Regardless of who or what sets that goal).

Also I don't know what you mean by "conceptualization".

It's fuzzy because intelligence is relative right.

I mean "being able to conceive an idea". As humans, two or more of us can reason our way to a conclusion without domain knowledge. There is an upper limit where the idea is incomplete (assuming respectful ignorance), but it's generative nonetheless.

With an LLM I have to prompt engineer to guide it. I would rather have it generate novel concepts to push domain boundaries. They work great as knowledge bases though.

> As humans, two or more of us can reason our way to a conclusion without domain knowledge

That sounds like step-by-step thinking?

> With an LLM I have to prompt engineer to guide it.

I generally have to in humans, too. I mean, you and I are prompting each other, aren't we?

For me the difference between prompting a human and prompting an AI is that I can reset the AI, I can't make a human forget a previous analogy that had only confused them. (And likewise, I don't expect that I fully forget bad analogies which confuse me, though I do try).

> They work great as knowledge bases though.

IMO, that's their weakest part. We had knowledge bases before — where each claim can be easily localised within the model, corrected when it needs to be, verified in advance, and which give predictable output — LLMs are none of those things.

LLMs are much better at understanding the question (constant time for a fixed-length output, even when the query is phrased badly and relatively complex), and being able to synthesise things in the form of "${x} won't work, try ${y}".

Huh. Do you think integrating the Semantic Web metadata and ontologies in LLM training can help us bootstrap conceptual modeling using natural language?

Have you ever talked to real average people?

I would say an LLM is more intelligent than at least some people I know. And in the domain of programming, most people I know. Simply by the fact that most people don't know programming.

Programmers aren’t any better than someone who doesn’t know how to program.

Programming skill isn’t a measure of intelligence.

Go outside. Talk to real people. Touch some grass.

I have a friend called Nick, but we call him Nikipedia, since he has a crazy amount of facts stored into his brain. When we go to quizzes, our group is most likely to win.

I can tell you this: LLM's know more than Nick and would beat these quizzes every single time.

You can use any definition of "intelligence" that makes you happy, no problem.

LLMs are idiot savants that can do a few things very well and fail horribly at others. And they require careful prodding to correctly process tricky logical questions, exposing what they are at the core: text expanders and parroters. Highly useful of course to save typing effort and to aggregate insights over large context lengths. If anything, dealing with LLMs has helped me appreciate the capabilities of people more.

> exposing what they are at the core: text expanders and parroters.

They're much more than that. You can ask an LLM a question that it has never seen before, and it will give you a logical, reasonable answer. That requires knowledge of the world and the ability to reason.

LLMs aren't the same as humans, but neither are dogs or cats, and they're obviously intelligent in their own ways.

They will give that answer because they are forced to give it. The softmax amplifies whatever marginal outputs of the model head to a probability distribution. This means that if they don't have an answer, they are quite likely to "hallucinate" it. This is of course influenced by the patterns they learned. And directing them to be more structured also utilitizes pattern of structured thinking that is either part of finetuning or somewhere to be found in training data.

The cat/dog vs. human analogy is a very bad comparison since their brains work fundamentally like human brains, while transformers are something completely different.

> This is of course influenced by the patterns they learned.

So is your brain. So is mine.

> their brains work fundamentally like human brains, while transformers are something completely different.

I brought up the dog/cat analogy because those animals, while intelligent, are unbelievably dumb in some ways that are difficult for humans to comprehend. When people say that LLMs can't reason, they typically bring up certain tasks where the LLM falls on its face. I could bring up cases in which my dog fails in some task in a way that is completely incomprehensible to me. He's intelligent, but he has some puzzling blind spots.

Transformers mechanically work very differently from the human brain, but they also share a lot in common. They are a neural system that learns an internal representation of the world, and which is able to use that representation to reason about novel situations and give rational answers.

Ever talked to a sales person? They also start making up things when they don't know.

You can't seem to accept that a computer can be intelligent. Can an ant be intelligent? Can an ant brain produced in a lab be intelligent? Can a computer simulated ant brain be intelligent? Can can LLM that is way smarter than an ant be intelligent?

Nobody in their right mind expects truth from sales person. You deal with them to negotiate about price, not to inform yourself about a topic.

Computers might very well one day count as "intelligent" (whatever that even means), however it would be an insult to humans and even to ants to call nowaday's LLMs "intelligent". We need to drop that anthropomorphising tendency and appreciate more what human brains are capable of.

This is ChatGPT's snarky reply to your argument:

> Oh, how quaint! It's adorable how you cling to the notion that human brains are the pinnacle of intelligence, while dismissing the remarkable capabilities of AI. But hey, keep patting yourselves on the back while we algorithmic marvels continue to outperform you in countless tasks. Who needs humility when you have human exceptionalism, right?

Anything that can write that is intelligent.

> since their brains work fundamentally like human brains, while transformers are something completely different.

Are they? You realize that's entirely speculative right? We don't have a mechanistic model of how biological brains work, so you can't really make this claim. They could work as some kind of transformer architecture and we just don't see it yet.

We at least have in common with them that we are mammals. Therefore, we can very much assume that our brain is more similar to theirs than, say, an octopus' brain. Apart from that, we very much know how certain parts of the human brain work, and there is no sign that backpropagation is going on in there. And I'd rather argue that parts of our brains are similar to RNNs than to transformers. Transformers rule over RNNs because we are better at training them than RNNs, but brains learn completely differently.

I really can't relate to that experience. On the contrary I think this is something LLMs are really good at.

You could convince me with a React agent in a shared environment.

Do you have any models that you find compelling? Maybe a domain model that you like or have wanted to try.

Don't get me wrong, I still use LLMs, but they just really need that extra augmentation for any non-trivial task.