News for hackers. Stuff that matters.
Re:What does Alan Kay think about LLMs? by ozten on Tuesday March 19, 2024 @03:30PM (#39759390)
> That humans also do this all the time is “interesting”, “dangerous” etc., but it is also why trying to move from superstition (this is actually what “reasoning by correlation” amounts to) to more scientific methods is critical for anything like civilization to be created.

"reasoning by correlation" as superstition is a brutal insight.

Re:What does Alan Kay think about LLMs? by kergonath on Tuesday March 19, 2024 @04:45PM (#39760026)
> "reasoning by correlation" as superstition is a brutal insight.

I don’t think he is right on that one, though. Reasoning by correlation is a kind of empiricism, and it can be tested. “These things happen together” or “if I do this, then that happens” can be disproven with statistical analysis even without any understanding of the underlying mechanisms at play their causes.

Superstition is beyond that; it is a belief that cannot be proven and that is not based on logic. The problem with superstition is not that there is no causal link between the supposed cause and the effect, it’s that there is not even a correlation.

Re:What does Alan Kay think about LLMs? by n4r9 on Tuesday March 19, 2024 @05:11PM (#39760251)
There are plenty of superstitions that are perfectly falsifiable. People continue to believe in them for comfort. For example, wearing odd socks to a sports match helps your team win. Or, leaving a fan on overnight might kill you.
Re:What does Alan Kay think about LLMs? by Phiwise_ on Tuesday March 19, 2024 @05:03PM (#39760190)
A road trip by random walk is also a kind of empiricism that can be tested. This does not imply it is worth the energy to actually do.
Re:What does Alan Kay think about LLMs? by sfpotter on Tuesday March 19, 2024 @06:05PM (#39760782)
A road trip by random walk is a valuable experience, the kind most people on HN lack.
Re:What does Alan Kay think about LLMs? by Phiwise_ on Tuesday March 19, 2024 @06:12PM (#39760839)
Call me crazy, but I don't think the goal of AI should be to give the rock with electricity in it "a valuable experience".
Re:What does Alan Kay think about LLMs? by sfpotter on Tuesday March 19, 2024 @07:33PM (#39761423)
What are you even talking about?
Re:What does Alan Kay think about LLMs? by Phiwise_ on Tuesday March 19, 2024 @08:05PM (#39761625)
The right way to build AI. You know, the topic of this thread. I've been wondering what you're talking about, though.
Re:What does Alan Kay think about LLMs? by halayli on Tuesday March 19, 2024 @04:58PM (#39760146)
Superstition is assuming a causation when you identify a correlation.
Re:What does Alan Kay think about LLMs? by brookst on Tuesday March 19, 2024 @05:14PM (#39760281)
Often followed by applying confirmation bias to see correlations when there are none.
Re:What does Alan Kay think about LLMs? by bsaul on Tuesday March 19, 2024 @07:05PM (#39761245)
i think this is the part of superstition. Reproductive, effective, correlation is the start of a scientific path to true discoveries. Superstition only begins when the correlation has never truly never been observed, but only repeated from generations to generations without any other kind of justification other than "that's what i heard".
Re:What does Alan Kay think about LLMs? by vinnyvichy on Thursday March 21, 2024 @07:24AM (#39777198)
There is also, fashionable on some message boards: assuming a lack of causation after only seeing the title of a report of correlation..

(One may also link fashion/accusations of fashionability etc etc to superstition..)

Re:What does Alan Kay think about LLMs? by kergonath on Thursday March 21, 2024 @01:51PM (#39781988)
Yes, some people confuse contrarianism for cynicism and both for wisdom.
Re:What does Alan Kay think about LLMs? by atoav on Tuesday March 19, 2024 @05:34PM (#39760483)
The problem here is that correlation is not a hard term for the people doing drawing the lines. People who take homeopathics for example will assure you they always get healthy when they took the sugar pills. What they fail to see is that they would have gotten healthy without it as well.

If you always rub your lucky penny before playing a sports game a superstitious person would account wins for the penny and excuse losses by not having rubbed the penny enough, the right way, not concentrated enough. This also feeds into correlation.

We are not talking about things actually being correlated, we are talking about people believing they are. Those are two wildly different things one is physical reality and doesn't change when you ignore it, the other is bound to individuals and tbeir world view and can take on absolutely ridiculous forms.

Re:What does Alan Kay think about LLMs? by alexvoda on Tuesday March 19, 2024 @05:15PM (#39760287)
I think the author and you are using different meanings for "reasoning".

We do know that many people even in professional fields confuse correlation with causation. And even when that doesn't happen, when only doing measurments, correlation is often considered good enough without giving any thought to the underlying mechanism. This may be the root of Goodhart's law stating that when a measure becomes a target it stops being a good measure. The measure was only correlated with the behaviour the person doing the measuring thought was measuring.

Superstition is similar, but drops any shred of statistical rigour, relying on mere anecdotal evidence.

Re:What does Alan Kay think about LLMs? by kurthr on Tuesday March 19, 2024 @04:17PM (#39759826)
It seems like the other part of this is the ability of LLMs to rationalize even answers that are not rational (he says piling BS on BS). That has previously been a very human (and quite distracting/confusing) activity and it's very effective on most people most of the time.
Re:What does Alan Kay think about LLMs? by codethatwerks on Tuesday March 19, 2024 @05:31PM (#39760453)
Causation is just correlation, and a good story. :-)
Re:What does Alan Kay think about LLMs? by ein0p on Tuesday March 19, 2024 @05:37PM (#39760513)
It’s great that he recognizes that most people aren’t to be trusted either. Most people outside the field are talking about “safety” as though humans never lie or make up bullshit, whereas in reality I trust GPT4 much more than I’d trust an average human, just not 100%. Literally people keep hyperventilating about “fake news” generation even though their entire news consumption already consists solely of pre approved narratives, many of which are fake, by omission or otherwise.

I can’t wait for the day when ultra long context LLMs are able to point out omissions and factual contradictions in human produced fake news for any given event

Re:What does Alan Kay think about LLMs? by ribosometronome on Tuesday March 19, 2024 @06:06PM (#39760787)
Are context windows not long enough to do this for most news articles and forum posts nowadays? I'd've thought the issue would be finding the relevant and reliable sources of truth for comparison would be the difficult part.
Re:What does Alan Kay think about LLMs? by ein0p on Tuesday March 19, 2024 @06:20PM (#39760899)
That is difficult, yes. The way the game is played is a handful of large outlets promote themselves as “trusted” and then drown out everything else as “untrustworthy”. But the fly in the ointment for them is that even these large/sophisticated opinion manipulators (they shouldn’t be called news sources at this point) are inconsistent both over time and sometimes also in the moment. Today’s public short attention span prevents it from seeing such inconsistencies or from paying attention to the broader picture. But guess what today’s models easily outperform humans on? The ability to pay attention to more than a small handful of things for longer than a few minutes.
Re:What does Alan Kay think about LLMs? by eska on Monday March 25, 2024 @04:05AM (#39813725)
The comparison is not between an LLM and an average human but an LLM and an expert on a subject.
Re:What does Alan Kay think about LLMs? by discreteevent on Tuesday March 19, 2024 @06:03PM (#39760771)
Here's an excellent recent comment along the same lines (in response to whether our intelligence is statistically based like LLMs):

> mjburgess 9 days ago

Suppose you touch a fireplace once, do you touch it again? No.

OK, here's something much stranger. Suppose you see your friend touch the fireplace, he recoils in pain. Do you touch it? No.

Hmm... whence statistics? There is no frequency association here, in either case. And in the second, even no experience of the fireplace.

The entire history of science is supposed to be about the failure of statistics to produce explanations. It is a great sin that we have allowed pseudosciences to flourish in which this lesson isnt even understood; and worse, to allow statistical showmen with their magic lanterns to preach on the scientific method. To a point where it seems, almost, science as an ideal has been completely lost.

The entire point was to throw away entirely our reliance on frequency and association -- this is ancient superstition. And instead, to explain the world by necessary mechanisms born of causal properties which interact in complex ways that can never uniquely reveal themselves by direct measurement.

https://news.ycombinator.com/item?id=39661963