Re:What Is ChatGPT Doing and Why Does It Work?
by funcantor on Wednesday February 15, 2023 @01:54AM
(#34800877)
I think its because it's sort of gone in the opposite way that science usually goes. Science traditionally been the process of physical phenomena that we don't understand => falsifiable theory that helps describe that phenomena.
Meanwhile, we started at stuff like the perceptron. The starting point was that we knew everything about that equation/classifier. Now we have a thing that we built from the ground up, and we don't fully grasp how it all comes together.
Re:What Is ChatGPT Doing and Why Does It Work?
by kqr on Wednesday February 15, 2023 @08:19AM
(#34803392)
> insisting on using a term directed at that level of generality implies a belief that nothing significant has happened since. Printing press? Just atoms. Cars? Just atoms. Computers? Just atoms.
It's very context-dependent but I don't read this as dismissive reduction, I read it as a statement of the incredible generalisability of atoms under the current laws of physics.
Re:What Is ChatGPT Doing and Why Does It Work?
by kilgnad on Wednesday February 15, 2023 @08:47AM
(#34803705)
Pfft more useless text written by a giant pile of atoms. Pointless.
Kidding aside, in the current context of human communication etiquette, to even refer to something as atoms outside of a demonstration or example is a deliberate dismissive reduction.
Re:What Is ChatGPT Doing and Why Does It Work?
by kqr on Wednesday February 15, 2023 @08:55AM
(#34803790)
I see the current context as something narrower, namely a discussion on the technical underpinnings of LLMs.
Re:What Is ChatGPT Doing and Why Does It Work?
by kilgnad on Wednesday February 15, 2023 @12:59PM
(#34807860)
The output of LLMs implies a higher level structure. You can't solely discuss the lower level underpinning unless you deliberately do not acknowledge that the output of LLMs are implying something else.
Re:What Is ChatGPT Doing and Why Does It Work?
by westoncb on Wednesday February 15, 2023 @02:24PM
(#34809199)
I think the fact that it's a technical discussion highlights why it's problematic: it might be appropriate for a total outsider who needs some kind of grasp of the basic underlying principle it operates on; but in discussion by a bunch of people with CS backgrounds, calling it merely a statical language generator erases distinction between it and other software of a prior age known not to be world-changing, overlooks recent innovations we don't yet have simple ways of referring to (largely because we don't understand emergent properties of LLMs)
It's like insisting on calling anything physical "atom collections". Yes, we get it, it's true (under a certain interpretation)—but it's clearly pointless to say except as an attempt at devaluing through reduction. (And it takes a particular stance on what it means to "be" something: to say it's literally the truth that anything physical is "just atoms" isn't the only way of looking at it.)
There were things we could've called "statistical word generators" decades ago; insisting on using a term directed at that level of generality implies a belief that nothing significant has happened since. Printing press? Just atoms. Cars? Just atoms. Computers? Just atoms.