News for hackers. Stuff that matters.
Re:The Unpredictable Abilities Emerging from Large AI Models by whimsicalism on Friday March 17, 2023 @03:58PM (#35202059)
I find the phrase "statistical analysis" a frustrating one nowadays as it seems to have become a signal for "I hold a particular philosophy of the mind".

I don't understand this use of "statistical" as a diminutive to describe these models.

Why can't incredibly complicated behavior be emergent from matrix multiplication subject to optimization in the same way that our biological matter has developed complicated emergent properties also being subject to optimization?

The loss function is very different, the optimization techniques as well, but the fundamental idea of complex behavior emerging out of a substrate subject to optimization seems common. I haven't seen a single good answer to that

Re:The Unpredictable Abilities Emerging from Large AI Models by mpalmer on Friday March 17, 2023 @05:05PM (#35202902)
I can't say that my philosophy affected my choice of words in the way that you mean. I'm just expressing wonder at how well the abstraction performs.
Re:The Unpredictable Abilities Emerging from Large AI Models by Jensson on Friday March 17, 2023 @05:14PM (#35202993)
> I find the phrase "statistical analysis" a frustrating one nowadays as it seems to have become a signal for "I hold a particular philosophy of the mind".

LLMs are trained to reproduce human text, that is different from for example AlphaGo that is trained to win Go games. Trained to reproduce data is what we mean with a statistical model, trained to win is how we got superhuman performance before, while trained to reproduce data performs worse than the original creators of the data.

Re:The Unpredictable Abilities Emerging from Large AI Models by pixl97 on Saturday March 18, 2023 @10:13AM (#35209284)
This is reductive in the sense is that things like AlphaGo were bad at Go for a very long time, and then with more compute power and algorithm changes suddenly they were far better. And the problem space for go is absolutely huge it is still an insignificant portion of the problem space for knowledge.
Re:The Unpredictable Abilities Emerging from Large AI Models by DanHulton on Friday March 17, 2023 @06:59PM (#35204025)
Well, to a degree, because we just don't do science like that.

You're the one supposing a thing, so the burden of proof is on you. You need to demonstrate that the "incredibly complicated behaviour" that you're referring to (I assume this is longhand for "thinking", but please correct me if I'm wrong) is indeed emerging from matrix multiplication. Especially given that what you're suggesting is unexpected, given the known way these models work and the explanations that have been put forth already that extrapolate from the known way these models work.

If science were so credulous as to accept the first proffered theory about a new development, well, we wouldn't have these interesting AI models in the first place!

Re:The Unpredictable Abilities Emerging from Large AI Models by YeGoblynQueenne on Saturday March 18, 2023 @09:56AM (#35209155)
Perhaps that's because you haven't asked a single good question?

What do matrix multiplication and optimisation have to do with the way the human mind, or the human brain work? That they have anything to do at all, is your assumption, that you seem absolutely convinced about- and then you go asking people why can't it be true? You say why it is true. It's your assumption.

Matrix multiplication and optimization are human mathematical techniques. They have about as good a chance of being something that exists in nature independently of humans as Magic: the Gathering and Call of Duty. They might be useful models to help us understand how things work, but to assume they are how things work is a huge leap of faith.

Ask the right questions and then an answer may even suggest itself. Ask questions that follow from your preconceived answers and you're in a world of fantasy.