News for hackers. Stuff that matters.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by layer8 on Friday August 16, 2024 @02:41PM (#41269144)
My impression is that LLMs “pattern-match” on a less abstract level than general-purpose reasoning requires. They capture a large number of typical reasoning patterns through their training, but it is not sufficiently decoupled, or generalized, from what the reasoning is about in each of the concrete instances that occur in the training data. As a result, the apparent reasoning capability that LLMs exhibit significantly depends on what they are asked to reason about, and even depends on representational aspects like the sentence patterns used in the query. LLMs seem to be largely unable to symbolically abstract (as opposed to interpolate) from what is exemplified in the training data.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by astromaniak on Friday August 16, 2024 @03:07PM (#41269386)
For some reasons LLMs get a lot of attention. But.. while simplicity is great it has limits. To make model reason you have to put it in a loop with fallbacks. It has to try possibilities and fallback from false branches. Which can be done on a level higher. This can be either algorithm, another model, or another thread in the same model. To some degree it can be done by prompting in the same thread. Like asking LLMs to first print high level algorithm and then do it step by step.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by layer8 on Friday August 16, 2024 @03:15PM (#41269436)
Iteration is important, but I don’t think that it can substantively compensate for the abstraction limitations outlined in the GP comment.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by refulgentis on Friday August 16, 2024 @04:27PM (#41270066)
> To make model reason you have to put it in a loop with fallbacks

Source? TFA, i.e. the thing we're commenting on, tried to, and seems to, show the opposite

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by cma on Friday August 16, 2024 @08:56PM (#41271605)
The comment may just be pattern matching on the topic, not reasoning about TFA.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by astromaniak on Saturday August 17, 2024 @12:43AM (#41272217)
When the task, or part of it, is np complete there is no way around. Model has to try all options till it find working one. In a loop. And this can be multi-step with partial fallback. That's how humans are thinking. They can see only to some depth. They may first determine promising directions. Select one, go dipper. Fallback if it doesn't work. Pattern matching mentioned is simplest one step solution. LLMs are doing it with no problems.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by llm_trw on Friday August 16, 2024 @05:35PM (#41270600)
LLMs get a lot of attention because they were the first architecture that could scale to a trillion parameters while still improving with every added parameter.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by Salgat on Friday August 16, 2024 @11:39PM (#41272073)
LLMs already do this. Their many wide layers allow for this, and as a final fallback, their output adjusts based on every token they generate (it's not all decided at once). All your statement really means is a vague "well it should do it more!" which yeah, is the goal of each iteration of GPT etc.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by arketyp on Friday August 16, 2024 @02:49PM (#41269215)
I agree. Although I wonder sometimes how much "about" is involved in my own abstract reasoning.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by kgeist on Friday August 16, 2024 @05:52PM (#41270719)
Aren't humans also prone to this?

Take the classic trick question, for example: "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?"

Most people give a wrong answer because they, too, "pattern match".

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by mecsred on Friday August 16, 2024 @06:00PM (#41270774)
I think you could reasonably describe that as "pattern matching at the wrong level". If you tell most people the first answer is wrong they will go up a level or two and work out the correct answer.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by nwhitehead on Friday August 16, 2024 @06:32PM (#41270980)
You would think so...

I asked this question in a college-level class with clickers. For the initial question I told them, "This is a trick question, your first answer might not be right". Still less than 10% of students got the right answer.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by adamisom on Friday August 16, 2024 @06:35PM (#41271002)
Students are more arrogant than LLMs (but still more generally intelligent)
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by roenxi on Friday August 16, 2024 @10:41PM (#41271926)
This idea that students are "more generally intelligent" requires specific arguments to support. ChatGPT alone has a vast breadth of knowledge that is impossible for a human to keep up with as well as superior skills to a student in any number of fields. I can find evidence that it has an IQ of around 124 [0] which is going to stretch most students (although in fairness the same article also speculates an IQ of 0).

Students can keep ahead of it with training in specific fields and it has a few weaknesses in specific skills but I think someone could make a reasonable claim that ChatGPT has superior general intelligence.

[0] https://medium.com/@soltrinox/the-i-q-of-gpt4-is-124-approx-...

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by chmod775 on Friday August 16, 2024 @11:27PM (#41272045)
> superior skills to a student in any number of fields

It's abhorrent any time we measure pure distilled intelligence.

When asked to come up with any non-basic novel algorithm and data structure, it creates nonsense.

Especially when you ask it to create vector instruction friendly memory layouts and it can't code in its preferred way. I had some fun trying to make it spit out a brute-force-ish solver for a problem involving basic orbital mechanics and some forces. Wouldn't even want try something more complicated. It can do generalized solvers somewhat, since it can copy that homework, but none that can express the kinds of terms you'd be working with (despite those also having code available in some research papers).

Speaking of which, it cannot even figure out some basic truths in orbital mechanics that can be somewhat easily derived from the formulas commonly given, nine times out of ten (you can get there if you're very patient and are able to filter its wrong answers).

But at the end of the day it was still a valuable tool to me as I was learning these things myself, since despite being often wrong, it nevertheless spat out useful things I could plug into Google to find more trustworthy sources that would teach me. Really neat if you're going in blind into a new subject.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by magic_hamster on Friday August 16, 2024 @11:04PM (#41271991)
Claiming that Chatgpt is more intelligent than a student is the same as saying an encyclopedia or a library is more intelligent than a student. Sure they retain more information. But Chatgpt is not AGI and it has no idea what it is even talking about.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ben_w on Saturday August 17, 2024 @02:00PM (#41276515)
> I can find evidence that it has an IQ of around 124

Despite me being someone who is generally impressed by the best LLMs, I think this says more about IQ tests than it does any AI.

Which isn't to shame those tests — we made those tests for humans, we were the only intelligence we knew of that used complex abstract language and tools until about InstructGPT — but it does mean we should reconsider what we mean by "intelligence".

My gut feeling is that a better measure is how fast we learn stuff. Computers have a speed advantage because transistors outpace synapses by the degree to which a pack of wolves outpaces continental drift (yes I did do the calculation), so what I mean here is how many examples rather than how many seconds.

But as I say, gut feeling — this isn't a detailed proposal for a new measure, it likely needs a lot of work to even turn this into something that can be a good measure.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by deafpolygon on Saturday August 17, 2024 @04:36AM (#41272832)
Knowledge != Intelligence
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by dinobones on Friday August 16, 2024 @07:24PM (#41271255)
This question becomes easy if you think about it algebraically/mathematically, but comically hard thinking about it intuitively, as in, trying to reason with language.

bat=ball+$1

(ball+$1)+ball=$1.10

2ball + $1 = $1.10

2ball = $0.10

ball = $0.05

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by amy-petrik-214 on Friday August 16, 2024 @06:19PM (#41270898)
10 cents! I'm 100% sure
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by SpaghettiCthulu on Friday August 16, 2024 @07:28PM (#41271274)
You are 100% wrong.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ignoramous on Friday August 16, 2024 @07:37PM (#41271302)
"A robot must obey orders given it by human beings..."

https://g.co/gemini/share/94238c5ed174 / https://archive.is/ZTkjI

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by card_zero on Friday August 16, 2024 @09:59PM (#41271799)
It might still say 5 cents if the problem was "the bat costs 90 cents more than the ball", since 5 cents is overwhelmingly the right answer to questions that look like this.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by qingcharles on Tuesday August 20, 2024 @06:00PM (#41304550)
That's a great conversation. I anthropomorphically feel bad for Gemini, in that it actually got a hard question right, and then it was gaslit into believing it was wrong.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by CooCooCaCha on Friday August 16, 2024 @07:45PM (#41271340)
“Don’t humans do this too?” Comes up in every thread about LLMs yet the conversation changes when you actually ask what you’d trust an LLM to do.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by SpicyLemonZest on Friday August 16, 2024 @11:28PM (#41272048)
I'm not saying this is you, but in my experience, the conversation changes because people flatly refuse to believe any concrete examples. I would and many companies do trust an LLM with low to medium skilled text processing tasks: internal knowledge sharing, first line customer support, low stakes document review. Github has their famous Copilot product, which I don't personally find too useful but many of my coworkers use.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by CooCooCaCha on Saturday August 17, 2024 @02:43PM (#41276868)
In other words, you don't trust it with anything important.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by SpicyLemonZest on Saturday August 17, 2024 @08:22PM (#41279162)
Really depends on what you understand the word to mean. All of the things I listed are “important” in the sense that they’ve gotta get done, and in the sense that it matters how well they’re done. Would I trust an LLM with something business-critical? No, but there’s quite a lot of people I’d put in the same boat.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by CooCooCaCha on Sunday August 18, 2024 @11:09AM (#41282820)
The difference is some humans exist that can be trusted with business critical tasks but no AIs exist with that level of competence.

This is actually a great example of what I'm talking about. All this talk about how useful AIs are, and how humans have flaws and yet the conclusion is always the same. AIs are only useful for tasks that are relatively easy and have a higher tolerance for failure.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by wavemode on Friday August 16, 2024 @09:14PM (#41271674)
When are we going to stop defending LLM intelligence on the grounds that "some people are stupid too"?

Human intelligence is not benchmarked by its lowest common denominators (just like how we don't judge LLM's on the basis of tiny 100M parameter models).

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by jacobsimon on Friday August 16, 2024 @09:21PM (#41271701)
The paper linked here starts off by asking whether LLMs can “solve complex problems in ways that resemble human thinking.” Why would we try to answer that question without discussing how humans think?
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by mrjin on Saturday August 17, 2024 @01:42AM (#41272344)
Coz we simply don't understand how we understand, period.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by mewpmewp2 on Saturday August 17, 2024 @02:15AM (#41272409)
So we shouldn't discuss it or try to understand it at all?
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by mrjin on Saturday August 17, 2024 @04:04AM (#41272720)
Where did you get that? You don't know you don't know. It's not going to sell if you literally say I don't know what we're doing...
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ben_w on Saturday August 17, 2024 @01:30PM (#41276206)
I'm a different person than you replied to, but I agree with them.

Your previous reply was to:

> Why would we try to answer that question without discussing how humans think?

And by replying to that with

> Coz we simply don't understand how we understand, period.

Well, when we don't understand how we understand, that is exactly when we should be discussing how humans think. Or at least how we think we think. And the bat and a ball example relates to research about how we think we think.

So yeah, your reply definitely comes across as saying the first half, "So we shouldn't discuss it", while finishing it with "period" also suggests "or try to understand it at all".

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by mannykannot on Saturday August 17, 2024 @08:40AM (#41274137)
> Why would we try to answer that question without discussing how humans think?

The post you are replying to does not suggest that we should. If anything, it is suggesting the opposite - that we should be considering the full range of human abilities (or at least those that are effective in solving complex problems) when addressing the question you have quoted.

I agree with most of what viraptor has said in this thread, but not in this particular case.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by p-e-w on Friday August 16, 2024 @10:04PM (#41271816)
When are we going to define what intelligence actually means, so these discussions can start making sense?

5 years ago, it was still common to read things like "AIs will never be as intelligent as mice, let alone humans", today, it's "sure, AIs are as intelligent as some humans, but not as intelligent as the right humans".

Noticed how everyone dropped the Turing Test like a hot potato as the gold standard for intelligence, the moment it became apparent that LLMs were about to pass it? Try to find a recent high-profile article invoking the Turing Test. Crickets. The intellectual dishonesty is nauseating.

The entire discussion is dominated by smart people who are scared shitless that AI is going to show them just how ordinary they are in the grand scheme of things.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by card_zero on Friday August 16, 2024 @10:39PM (#41271918)
Do we still have Turing Test competitions? Now that everybody's used to LLMs, and detecting their style, it might be interesting. I'm not sure the LLMs would do very well against humans who were experienced and alert.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by viraptor on Saturday August 17, 2024 @01:33AM (#41272323)
"detecting their style" is trivial only because people take the default output and run with it. You can significantly alter it if you want and it would be an obvious thing to do for such test. Right now the default output styles of public LLMs have no reason to try avoiding detection and is not trained to do it.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by card_zero on Saturday August 17, 2024 @09:47AM (#41274568)
I didn't exactly mean superficial style, but properties like creativity, liveliness, and consistency. I hesitate to say that these are things LLMs can't do, because I'll provoke somebody into telling me that they totally can, but weaknesses like this could be ways for human intuition to detect an LLM. It might have to be a long competition, because an unusual set of prompts (or training, even) is likely to initially impress the humans: time should be allowed for novelty to wear off.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by viraptor on Saturday August 17, 2024 @10:10AM (#41274718)
> properties like creativity, liveliness, and consistency

I don't believe we train The LLMs for those things specifically. It will be interesting to see if some datasets for this appear. I think we can still make huge improvements by just caring about that more.

We've got Samantha though, so I hope we will see those attributes too. https://erichartford.com/meet-samantha

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by BriggyDwiggs42 on Friday August 16, 2024 @11:23PM (#41272031)
Wait we say LLMs are as smart as some humans? I must have missed the memo.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by p-e-w on Friday August 16, 2024 @11:39PM (#41272072)
I mean LLMs crush most humans (even many professionals) at coding/legal/medical/etc. exams.

Ah, wait. That only counts as evidence of intelligence when humans do it, right?

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by wavemode on Saturday August 17, 2024 @12:01AM (#41272119)
Those same humans the AI beats at one task, the human would beat it at 1000 others. So the human is 1000x more intelligent!

Ah, wait. That only counts as evidence of intelligence when AIs do it, right?

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ben_w on Saturday August 17, 2024 @01:51PM (#41276421)
It's the other way around. Any one human will only beat an AI at two or three tasks — the first two being our day job and our mother tongue, the third possibly being our hobby — while the AI will out-perform us at thousands of others tasks.

Only if you count humanity as a whole do we beat an LLM at everything.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by wavemode on Saturday August 17, 2024 @02:35PM (#41276791)
There are two logical problems with what you've written here.

1. You're counting "our day job" as one task, while counting all individual prompts an AI can answer as each being their own task. This is obviously misleading (and I could just as easily perform the same compression in the opposite direction - chatbots can only do one task: "be a chatbot").

2. You're not controlling for training. It's already meaningless to compare the intelligence of one entity trained to do a task with another entity that was not trained to do that task.

But even ignoring those fallacies, what you've written here is still not true. 99% of things an LLM could supposedly "out-perform" a human at, the human would actually outperform if you provided that human with the same text resources the LLM used to conjure its answer. Regurgitating facts is not evidence of intelligence, and humans can do it easily (and do it without hallucinating, which is key) if you just give them access to the same information.

But when you go in the opposite direction, achieving parity is no longer so simple. When LLM's fail to do math, fail to strategize, fail to process rules of abstract games, etc. there is no textbook or lecture or article you can provide to the LLM that magically makes the problem go away. They are fundamental limitations of the AI's capabilities, rather than just a result of not possessing enough information.

What many people don't seem to realize is, when merely getting up in the morning and brushing your teeth you are already exercising more intelligence than any AI has ever possessed. (Anyone who has ever worked in robotics or visual processing can attest to that enthusiastically.) So don't even get me started on actual critical thinking.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ben_w on Saturday August 17, 2024 @02:52PM (#41276964)
Neither of those is a logical problem.

1. This is indeed a simplification, but for any single task in your day job, it would be those tasks where you have the most experience. For example, I used to write video games, AI does a better job of game design than me, but I'm the better programmer.

2. Unimportant, as the consideration I was rejecting was performance in tasks.

As it happens, some of my other recent messages demonstrate that I agree they are low intelligence for this exact reason.

> 99% of things an LLM could supposedly "out-perform" a human at, the human would actually outperform if you provided that human with the same text resources the LLM used to conjure its answer

Could I pass a bar exam of a medical exam, by reading the public internet, with no notes and just from memory, which is what a base model does?

Nope.

Could I do it with a search engine, which is what RAG assisted LLMs do?

Perhaps.

> humans can do it easily (and do it without hallucinating

Hell no we mess that up almost constantly.

> When LLM's fail to do math, fail to strategize, fail to process rules of abstract games, etc. there is no textbook or lecture or article you can provide to the LLM that magically makes the problem go away

I'm sure I've seen this done. I wonder if I'm hallucinating that certainty…

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by wavemode on Saturday August 17, 2024 @03:03PM (#41277055)
> Could I pass a bar exam of a medical exam, by reading the public internet, with no notes and just from memory, which is what a base model does? Nope.

You're -still- ignoring the fact that these models spend millions of GPU-hours in training. I'm sure you could manage.

> Hell no we mess that up almost constantly.

"Almost constantly"? Is this satire? I'd fire any such person, and probably recommend them psychiatric treatment.

AI-hype people really think so little of human beings? I certainly hope my pilot isn't "almost constantly" hallucinating his aviation training.

> I'm sure I've seen this done. I wonder if I'm hallucinating that certainty…

Show me the conversation.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by BriggyDwiggs42 on Saturday August 17, 2024 @12:25AM (#41272179)
Wait are LLMs crushing professionals at coding? Anyways, LLMs are clearly better than humans at memorizing information, and they most certainly have some ability to retrieve and adapt this information to solve problems. On tests, they can do some degree of generalization to answer questions with different wording from their training data. However, because humans are comparably quite bad at memorizing, we know that when humans do well on medical tests, it usually (not always) means that they “understand” how, for example, anatomy works. They’ve formed some model in their head of how to break the body into parts and parts into subparts and how parts interact and so on because doing so is actually easier for most humans than trying to remember the linguistic facts.

Understanding in this sense seems different from the memorizing+flexible retrieval we know LLMs exceed at because it extends much further beyond its training distribution. If I ask a (good) medical professional a question unlike anything they’ve ever seen before, they’ll be able to draw on their “understanding” of anatomy to give me a decent guess. LLMs are inconsistent on these kinds of questions and often drop the ball.

We can also point to training data requirements as a discrepancy. In brain organoid experiments, we observe a much lower quantity of examples are required to achieve neural-network-like results. This isn’t surprising to me. Biological neurons have exceedingly complex behavior; it takes hundreds of neural network nodes to recreate the behavior of a single neuron, and they can reorganize their network structure in response to stimuli, form loops, operate in non-discretized time, etc. We don’t know how far transformers will be able to go, but I think that if you want to make a model that holds a candle to that sort of complexity, you’ll need at least many, many orders of magnitude more scale or, more likely, a different, less limiting architecture.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by mewpmewp2 on Saturday August 17, 2024 @02:17AM (#41272416)
Most people can't code at all fyi.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by BriggyDwiggs42 on Saturday August 17, 2024 @03:16AM (#41272584)
Most professional coders can, in fact, code.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by mewpmewp2 on Saturday August 17, 2024 @10:48AM (#41274962)
Most people are not professional coders.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by TeMPOraL on Saturday August 17, 2024 @11:11AM (#41275092)
And to short-circuit the argument: approximately no one is, at the same time, a professional coder, writer, artist, detective, translator, doctor, and <insert one or ten or a hundred more occupations here>. GPT-4 is all of them at once, and outperforms an average professional of any of those occupations in their respective field.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by BriggyDwiggs42 on Saturday August 17, 2024 @05:12PM (#41278006)
You said it outperforms the average professional in any of these occupations (including detective and artist?) and more? That’s a very bold claim. Can you substantiate that somehow? I don’t disagree for a second that incrementally better LLMs + robotics will be able to automate a large portion of labor, but that doesn’t in my eyes make them smarter than humans. Jobs today aren’t exactly the fullest realization of human potential; you wouldn’t call a robotic arm smarter than a human for being better on the line.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ithkuil on Saturday August 17, 2024 @01:07PM (#41275983)
I thought this thread was an answer to the qualifier in parenthesis:

> I mean LLMs crush most humans (even many professionals) at coding/legal/medical/etc. exams.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ben_w on Saturday August 17, 2024 @01:44PM (#41276347)
> Wait are LLMs crushing professionals at coding?

Exams, yes. Not the work yet. This is also why they've not made doctors and lawyers obsolete: even in the fields where the models perform the best, you're getting the equivalent of someone fresh out of university with no real experience.

I suspect you're right about your point with generalise vs. memorise. Not absolutely sure, but I do suspect so.

I also suspect we'll get transformative AI well before we can train any AI with as few examples as any organic brain needs. Unless we suddenly jump to that with one weird trick we've been overlooking this whole time, which wouldn't hugely surprise me given how many radical improvements we've made just by trying random ideas.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ben_w on Saturday August 17, 2024 @01:37PM (#41276284)
> Noticed how everyone dropped the Turing Test like a hot potato as the gold standard for intelligence, the moment it became apparent that LLMs were about to pass it? Try to find a recent high-profile article invoking the Turing Test. Crickets. The intellectual dishonesty is nauseating.

Not quite that bad, but I hear you.

https://arxiv.org/html/2405.08007v1

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by Kiro on Saturday August 17, 2024 @02:38AM (#41272466)
Just because you're tired of hearing something doesn't make it less true. "When are we going to stop" is a manipulative argument.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ben_w on Saturday August 17, 2024 @01:36PM (#41276265)
> Human intelligence is not benchmarked by its lowest common denominators (just like how we don't judge LLM's on the basis of tiny 100M parameter models).

Indeed, but we do also catalogue our cognitive biases — kinda the human version of what are now called hallucinations when LLMs do them. (When Stable Diffusion does it, it's "oh god the fingers").

It's worth caring about both strengths and weaknesses.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by fragmede on Saturday August 17, 2024 @01:45PM (#41276355)
Yes it is. Warning labels on products and laws aren't written with genius level people in mind, but the lowest common denominator. Tv politics is the way is it because of said lowest common denominator of intelligence. It's not some people are dumb too, most people are dumb in the wrong contexts, even you and I pre-coffee or sleep deprived or wasted drunk or high.

IQ tests are averaged out so 100 is average intelligence, not the high point.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by bad_user on Friday August 16, 2024 @11:52PM (#41272096)
People pattern match, but people also apply logic and abstract thinking, far surpassing the abilities of LLMs, and this is a fundamental limitation that won't get fixed by more processing power and training data.

The big difference in behavior is in how people or LLMs approach new problems. A LLM is incapable of solving a problem that's similar to one that it was trained on, but with slightly changed requirements. A LLM is also incapable of learning, once you point out their error, or of admitting that it doesn't know.

Regarding people, I find it interesting that even lower IQ people are capable of tasks that are currently completely out of reach for AI. It's not just the obvious, such as self-reflection, but even tasks that should've been solved by AI already, such as driving to the grocery store.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by viraptor on Saturday August 17, 2024 @01:20AM (#41272295)
There's lots of assumptions here that we've got examples to disprove:

> but people also apply logic and abstract thinking

Which people? If we universally did that as a default, elections would look massively different.

> A LLM is incapable of solving a problem that's similar to one that it was trained on, but with slightly changed requirements

Getting good answers for coding questions on my private code/databases disproves this. The requirements have been changed significantly. I've been through ~20 turn chat with LLM investigating a previously unseen database, suggesting queries to get more information, acting on it to create hypotheses and follow up on them.

> A LLM is also incapable of learning, once you point out their error,

This is the standard coding agent loop - you feed back the error to get a better answer through in-context learning. It works.

> or of admitting that it doesn't know.

Response from gpt: «I apologize, but I'm not able to find any reliable information about a person named "Thrust Energetic Aneksy."»

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by Jensson on Saturday August 17, 2024 @03:08AM (#41272555)
> Response from gpt: «I apologize, but I'm not able to find any reliable information about a person named "Thrust Energetic Aneksy."»

The model says that because it is trained to say that to specific queries. They have given it a lot of prompts with "Who is X" and showing the responses "I don't know about that person".

The reason you don't see "I don't know" much in other kinds of problems is that there it isn't easy to create such data examples where the model says I don't know while still making the model solve problems that are in its dataset, since it starts to pattern match all sort of math problems to "I don't know" even when it could solve it.

A human can look at his own thoughts and realize how he is solving it, and thus knows a lot better what he knows and not. LLMs aren't like that, they don't really know what they do know. The LLM doesn't know its own weights when it picks a word, it has no clue how certain it is, it just predicts whether a human would have said "I don't know" to the question, not whether itself would know.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by viraptor on Saturday August 17, 2024 @04:56AM (#41272901)
> LLMs aren't like that, they don't really know what they do know.

They actually do. The information is saved there, you just need to ask explicitly, because the usual response doesn't expose it. (But likely can be fine tuned to do that) https://ar5iv.labs.arxiv.org/html/2308.16175

Also the original claim was that they're not capable of responding with "I don't know", so that's what I was addressing.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by Jensson on Saturday August 17, 2024 @03:04AM (#41272544)
> I've been through ~20 turn chat with LLM investigating a previously unseen database, suggesting queries to get more information, acting on it to create hypotheses and follow up on them.

You do know that when that happens the LLM usually just throws random stuff at you until you are happy? That is much easier to do than to reason, LLM solved the much easier problem of looking smart than being smart, trick is to make the other person solve the problem for you while attributing it to you.

You see humans do this as well in hiring interviews etc, it is really easy to trick people who want you to succeed.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by viraptor on Saturday August 17, 2024 @04:01AM (#41272707)
When the hypotheses make sense, the tests for them work and the next steps are based on the previous test results that's not "random stuff". I made that explicit in the first message. This problem wasn't solvable with random guesses.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by photonthug on Saturday August 17, 2024 @02:00AM (#41272379)
> This is the standard coding agent loop - you feed back the error to get a better answer through in-context learning. It works.

This looks like it works sometimes, but only if "pointing out the error" is coincidentally the same as "clarifying problem spec". Admittedly for really simple cases those are the same or hard to tell apart. But it always seems clear that adding error correction context is similar to adding additional search terms to get to a better stack-overflow page. This feels very different than updating any kind of logical model for the problem representation.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by viraptor on Saturday August 17, 2024 @02:13AM (#41272401)
It doesn't have to be as explicit as an extra term. The error feedback can be a failing test which was just modified, or a self-reflection result from a prompt like "does what you created satisfy the original request?".

Updating the logical model of the problem also happens when you do a database investigation I mentioned earlier. There's both information gathering and building on it to ask more specific questions.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by bad_user on Tuesday August 20, 2024 @11:21AM (#41300868)
> Which people? If we universally did that as a default, elections would look massively different.

All people, otherwise we wouldn't be able to do basic tasks, such as finding edible food or recognise danger.

I dislike how you discount the way other people vote as being somehow irrational, while I'm sure you consider your own political thinking as being rational. People always vote according to their own needs and self-interest, and in terms of politics, things are always nuanced and complicated. The fact that many people vote contrary to your wishes is actually proof that people can think for themselves.

> Getting good answers for coding questions on my private code/databases disproves this.

I use GitHub Copilot and ChatGPT every day. It only answers correctly when there's a clear, and widely documented pattern, and even then, it can hallucinate. Don't get me wrong, it's still useful, but it shows in no way an ability to reason, or the capacity to admit that a solution is out of its reach.

Your experience with coding is kind of irrelevant to the question at hand.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by make_it_sure on Saturday August 17, 2024 @02:01AM (#41272384)
you're so off, the comment below gives some good points
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by FeepingCreature on Saturday August 17, 2024 @02:39AM (#41272469)
> The big difference in behavior is in how people or LLMs approach new problems. A LLM is incapable of solving a problem that's similar to one that it was trained on, but with slightly changed requirements. A LLM is also incapable of learning, once you point out their error, or of admitting that it doesn't know.

This is simply not true.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by dTal on Saturday August 17, 2024 @05:17AM (#41273002)
Indeed, domain-specific reasoning ability is a seminal psychology result: people are in general sucky at a specific type of logic puzzle, unless you frame it as enforcing a social rule - then they're excellent.

https://en.wikipedia.org/wiki/Wason_selection_task

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by gcanyon on Saturday August 17, 2024 @08:14AM (#41273943)
Since I've known this puzzle since childhood, my "pattern match" answer is the correct answer. I wonder if someone took it up a level and posed a puzzle that looked like this puzzle but had some other subtle complication, whether it would fool me.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by Balgair on Saturday August 17, 2024 @09:01AM (#41274260)
> Aren't humans also prone to this?

Actually, no. We can do 'real' reasoning and come up with novel conclusions. In fact, we have such an example going all the way back to Plato in Meno, about 2300 years ago. It's the doubling of the square dialog of Socrates and the slave boy.

https://classics.mit.edu/Plato/meno.html (Crtl+F for 'square')

Really, the whole dialog is relevant to this discussion with LLMs knowing things.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by slashdave on Friday August 16, 2024 @08:41PM (#41271545)
Given that LLMs are trained on tokens and not symbols, that is, they have no context or world model, I would say that LLMs cannot symbolically abstract, given that they have neither a mechanism nor the data to do so.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by Salgat on Friday August 16, 2024 @11:32PM (#41272056)
I wonder if reasoning is intrinsically tied to the need for a model to be able to perform well with extremely limited training data. Current LLMs require the sum total of all human knowledge to work correctly (even a few decades ago, there was no where near enough data to train one to be useful, regardless of available computing), meanwhile humans only need access to a few books and conversations with uneducated people growing up to achieve brilliant ideas.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by ekianjo on Saturday August 17, 2024 @12:28AM (#41272187)
> Current LLMs require the sum total of all human knowledge to work correctly

the total sum of human knowledge is not found in the digital world.

Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by Salgat on Saturday August 17, 2024 @10:32AM (#41274872)
I'd argue the vast majority of it is, from newspaper articles spanning back centuries to the vast majority of books written, down to every conversation someone has had on public social media websites such as Reddit.
Re:Does Reasoning Emerge? Probabilities of Causation in Larg... by xkcd1963 on Saturday August 17, 2024 @03:03AM (#41272543)
A unicellular organism alone has already more variables to deal with than an LLM, now multiply that by the amount of cells a brain has.