I just gave up on using SwiftUI for a rewrite of a backend dashboard tool.
The LLM didn't give up. It kept suggesting wilder, and less stable ideas, until I realized that this was a rabbithole full of misery, and went back to UIKit.
It wasn't the LLM's fault. SwiftUI just isn't ready for the particular functionality I needed, and I guess that a day of watching ChatGPT get more and more desperate, saved me a lot of time.
But the LLM didn't give up, which is maybe ot-nay oo-tay ight-bray.
Hold on a minute I was under the impression that "reasoning" was just marketing buzzword the same as "hallucinations", because how tf anyone expected GPUs to "reason" and "hallucinate" when even neurology/psychology don't have a strict definition of those processes.
I don't have a strict enough definition to debate if this reasoning is "real" - but from personal experience it certainly appears to be performing something that at least "looks" like inductive thought, and leads to better answers than prior model generations without reasoning/thinking enabled.
Am I oversimplifying it? Is everybody else over-mystifying it?
If you allow me to view the weights of a model as the axioms in an axiomatic system, my (admittedly limited) understanding of modern "AI" inference is that it adds no net new information/knowledge, just more specific expressions of the underlying structure (as defined by the model weights).
So while that does undercut my original flippancy of it being "nothing but gradient descent" I don't think it runs counter to my original point that nothing particularly "uncanny" is happening here, no?
The claims about LLM reasoning are precisely related to this point. Do LLMs follow some internal deductive process when they generate output that resembles such a logical process to humans? Or are they just producing text that looks like reasoning, much like an absurdist play might do, and then simply picking a conclusion that resembles other problems they've seen in the past?
I don't think any arguments about the base nature of the model are particularly helpful here. In principle, deductive reasoning can be expressed as a mathematical function, and for any function, there is a neural net that can approximate it with arbitrary precision. So it's not impossible that the model actually does this, but it's also not a given - this first principles approach is just not helpful. We need more applied study of how the model actually works to probe this deeper.
Selling an algorithm that can write a list of steps as reasoning is bordering on fraud.
It's not uncommon that they guess the right solution, and then "reason" their way out of it.
"""Reasoning: The deduction of inferences or interpretations from premises."""
Sounds like any logic program to me?
Apparently from Latin, ratiō, which has meanings including "explanation" and "computation"?
(The Apple paper has had many serious holes poked in it.)
I also find the assumption that tech-savvy individuals would inherently be for what we currently call AI to itself, be weird. Unfortunately I feel as though being knowledgable or capable within an area is conflated with an over-acceptance of that area.
If anything, the more I've learned about technology, and the more experienced I am, the more fearful and cautious I am with it.
On the other hand, most of the "blindly accept every new technique or gadget" folks I know are also tech workers so maybe there's nothing going on there. Wonder if there's study on this
And even if the two could theoretically be separated, how sure are we that these AI agents are in that category? I'm pretty sure they are neither, but that doesn't mean it isn't a pretty abhorrent possibility that should be addressed.
You should re-think the bias that led you to this belief.
(FWIW, I do think there's some very unhealthy attitudes to AI and LLMs going around, like people feel the only two options are 'the singularity is coming' and 'they're useless scams', which tends to result in a large quantity of bullshit on the topic)
No, you are turning a few loud people into a false dichotomy. The vast majority of people are somewhere between "LLMs are neat" and "I don't think LLMs are AGI"
The vast majority of people do not comment. Using only the comments that people go out of their way to make as your data source is a huge sampling error.
That is just plain wrong, as anybody who spent more than 10 minutes with a LLM within the last 3 years can attest. Give it a try, especially if you care to have an opinion on them. Ask an absurd question (that can be, in principle, answered) that nobody has asked before and see how it performs generalizing. The hype is real.
I'm interested what study you refer to. Because I'm interested in their methods and what they actually found out.
The crux is that beyond a bit of complexity the whole house of cards comes tumbling down. This is trivially obvious to any user of LLMs who has trained themselves to use LLMs (or LRMs in this case) to get better results ... the usual "But you're prompting it wrong" answer to any LLM skepticism. Well, that's definitely true! But it's also true that these aren't magical intelligent subservient omniscient creatures, because that would imply that they would learn how to work with you. And before you say "moving goalpost" remember, this is essentially what the world thinks they are being sold.
It can be both breathless hysteria and an amazing piece of revolutionary and useful technology at the same time.
The training set argument is just a fundamental misunderstanding, yes, but you should think about the contrapositive - can an LLM do well on things that are _inside_ its training set? This paper does use examples that are present all over the internet including solutions. Things children can learn to do well. Figure 5 is a good figure to show the collapse in the face of complexity. We've all seen that when tearing through a codebase or trying to "remember" old information.
beta api's so its moving waters but that's my thoughts after playing with it, the paper makes much more sense in that context
In my experience, its better to simply try using LLMs in areas where they don't have a lot of training data (e.g. reasoning about the behaviour of terraform plans). Its not a hard cutoff of being _only_ able to reason exactly about solved things, but its not too far off as a first approximation.
The researchers took exiting known problems and parameterised their difficulty [1]. While most of these are not by any means easy for humans, the interesting observation to me was that the failure_N was not proportional to the complexity of the problem, but more with how common solution "printouts" for that size of the problem can typically be encountered in the training data. For example, "towers of hanoi" which has printouts of solutions for a variety of sizes went to very large number of steps N, while the river crossing, which is almost entirely not present in the training data for N larger than 3, failed above pretty much that exact number.
[1]: https://machinelearning.apple.com/research/illusion-of-think...
On the one hand, it's sort of like red teaming. On the other hand, it clearly gives consumers a false sense of ability.
The "AI isn't really intelligence" argument is so tired now it has a whole Wikipedia page about it: https://en.m.wikipedia.org/wiki/AI_effect
Researchers in the field used to acknowledge that their computational models weren't anywhere close to AI. That all changed when greed became the driving motivation of tech.
Where is everyone getting this misconception? I have seen it several times. First off, the study doesn't even try to qualify whether or not these models use "actual reasoning" - that's outside of the scope. They merely examine how effective thinking/reasoning _is_ at producing better results. They found that - indeed - reasoning improves performance. But the crucial result is that it only improves performance up to a certain difficulty-cliff - at which point thinking makes no discernable difference due to a model collapse of sorts.
It's important to read the papers you're using to champion your personal biases.
LLM's aren't magic - those who claim they are are hyping for some reason or another. Ignore them. View AI objectively. Ignore your bias.
I would agree with you if this were about inventing the algorithm for itself - it may well be that you'd need some amount of visual reasoning to come up with it. But that's not what the GP (or the paper) were talking about.
You're oversimplifying the results a bit here. They show that reasoning decreases performance for simple problems, improves performance for more complex ones, and does nothing for very complex problems.
The LLMs don't "reason" by any definition of the term. If they did, then the Tower of Hanoi and the river problem would have been trivial for them to handle at any level because ultimately the solutions are just highly recursive.
What the LLMs do is attempt to pattern match to existing solved problems in their training set and just copy those solutions. But this results in overthinking for very simple problems (because they're copying too much of the solutions from their training set), works well for the somewhat complex problems like a basic Tower of Hanoi, and not at all for the problems that would require actual reasoning because...they're just copying solutions.
The point of the paper is that what LLMs do is not reasoning, however much the AI industry may want to redefine the word to suit their commercial interests.
The Apple study that did Towers of Hanoi and concluded that giving up when the answers would have been too long to fit in the output window was a sign of "not reasoning"?
https://xcancel.com/scaling01/status/1931783050511126954
I mean, on that basis, anyone who ever went "TL;DR" is also demonstrating that humans don't reason.
> That's not "intelligence" however you define it because they can only solve things that are already within their training set.
This is proven untrue by, amongst other things, looking at them playing chess. They can and do play moves not found in the training data: https://www.lesswrong.com/posts/yzGDwpRBx6TEcdeA5/a-chess-gp...
In this case, it would have been better for the AI industry if it had been 700 programmers, because then the rest of the industry could have argued that the utter trash code Builder.ai generated was the result of human coders spending a few minutes haphazardly typing out random code, and not the result of a specialty-trained LLM.