News for hackers. Stuff that matters.
Re:The "it" in AI models is the dataset by jstummbillig on Thursday April 25, 2024 @05:32AM (#40155313)
Why does it make you sad? It seems intuitiv and simple. And in reality of course the optimisation part is not trivial. What would we better if the "it" was more complicated?
Re:The "it" in AI models is the dataset by michaelt on Thursday April 25, 2024 @06:21AM (#40155622)
It used to be that people would get into these fields thinking ML would need specifically human insights, deep thinking, and philosophical insights about the nature of consciousness.

You would get into natural language modelling because you had a deep love of language. Because you think you're close to figuring language out in a systematic way, with just a few years more study.

There's a certain sadness, I think, in the revelation that the robots don't need the expertise of humanity's greatest experts and masters, they just need us to click all the squares that contain a motorcycle.

Re:The "it" in AI models is the dataset by sevagh on Thursday April 25, 2024 @08:16AM (#40156563)
> It used to be that people would get into these fields thinking ML would need specifically human insights, deep thinking, and philosophical insights about the nature of consciousness.

What's sadder is coming into a field pre-deciding that the way you approach it "is the right way" and can't tolerate that different mindsets can also get results.

Re:The "it" in AI models is the dataset by disgruntledphd2 on Thursday April 25, 2024 @06:30AM (#40155696)
This is 100% not why I am sad, see my other reply for information.

As an aside, it's wild how people put their own spin onto what I said.

Obviously I should have been clearer :shrug:.

Re:The "it" in AI models is the dataset by xanderlewis on Thursday April 25, 2024 @06:45AM (#40155803)
Well, you have to forgive some for making assumptions based on your choice of username…
Re:The "it" in AI models is the dataset by disgruntledphd2 on Thursday April 25, 2024 @06:58AM (#40155891)
Fair, I'm just generally disgruntled to be fair, the PhD was just the name of my soon abandoned blog.
Re:The "it" in AI models is the dataset by nicklecompte on Thursday April 25, 2024 @07:54AM (#40156345)
> in the revelation that the robots don't need the expertise of humanity's greatest experts and masters

If it's any consolation, this is more of a delusion than a revelation. Throwing gigantic piles of data at extremely primitive artificial neural networks[1] succceeds at artificial benchmarks and might get OpenAI over the line of commercial viability. But GPT still doesn't understand what numbers are, either in text or visually. Sora doesn't understand causality or object permanence. DALL-E doesn't have a clue what a dog or a cat actually is (despite the misleading claim in the post).

"Scaling is all you need" will be an ironic slogan of the upcoming AI winter. These machines have read more text and watched more videos than 1,000 humans could do in their lifetimes... and they're still dramatically dumber than mice.

[1] "primitive" relative to biological neurons, obviously they are quite fancy relative to other software.

Re:The "it" in AI models is the dataset by xanderlewis on Thursday April 25, 2024 @06:26AM (#40155661)
How do you know? We’re not there yet.
Re:The "it" in AI models is the dataset by disgruntledphd2 on Thursday April 25, 2024 @06:29AM (#40155686)
Because of the way it's presented, as if it's some vast new discovery that OpenAI have made, rather than common wisdom.

It makes me sad when people rediscover things (with massive compute in this case), that were already known.

It's very much spend a year in the lab to save an hour in the library.

Re:The "it" in AI models is the dataset by macilacilove on Thursday April 25, 2024 @06:09AM (#40155547)
Possibly because being in the business of trying to turn iq edge into money, not data edge into money.