News for hackers. Stuff that matters.
Re:Every model learned by gradient descent is approximately ... by vasilipupkin on Saturday February 24, 2024 @10:50PM (#39497433)
bingo, I think it's a weird gotcha. Most human occupations require years of training exactly for this purpose, so that humans can just regurgitate data they have seen in training. Very few humans ever produce truly novel insights. you wouldn't hire a divorce lawyer for a corporate merger and vice versa
Re:Every model learned by gradient descent is approximately ... by jjmarr on Saturday February 24, 2024 @11:11PM (#39497549)
A (common law) lawyer has to remember and understand case law, but they also have to create analogies to explain why and how new situations are like older ones. That's one of the main parts of being a good lawyer.

AI will be disruptive to aspects of professions that rely on rote memorization or research on a large corpus of data. But many of these supposedly memorization heavy professions require the ability to go beyond one's training and extrapolate from what one remembers.

Re:Every model learned by gradient descent is approximately ... by vasilipupkin on Sunday February 25, 2024 @08:57AM (#39500859)
Absolutely, but I think my example still stands

If humans could come up with truly novel solutions frequently, it would be no problem hiring a divorce lawyer to do a corporate merger. But there is a very low likelihood of this ever happening anywhere.

Re:Every model learned by gradient descent is approximately ... by dartos on Saturday February 24, 2024 @11:01PM (#39497493)
I think that’s a gross oversimplification.

Have you been a plumber or an electrician?

Different buildings require different, occasionally novel, solutions. Not all are textbook (most aren’t.) Not all innovations are published or even widely communicated.

Just like in programming, the devil is in the details, otherwise we’d all be using ruby in rails for our web servers, since regurgitating how to build a CRUD app is all you’d need.

Re:Every model learned by gradient descent is approximately ... by BoiledCabbage on Saturday February 24, 2024 @11:16PM (#39497575)
> Different buildings require different, occasionally novel, solutions. Not all are textbook (most aren’t.) Not all innovations are published or even widely communicated.

True, but it's mostly derivative just like LLMs. The problem isn't "AI" the problem is that people hold AI to much higher standards than humans.

AI to them means "scifi", omniscient and omnipotent. You can have AI and it still be flawed, have weaknesses and shortcomings just like people do.

Re:Every model learned by gradient descent is approximately ... by card_zero on Sunday February 25, 2024 @12:01AM (#39497794)
So AI is unoriginal and thick as pigshit and is therefore an accurate simulation of a human, neat argument.
Re:Every model learned by gradient descent is approximately ... by dartos on Sunday February 25, 2024 @01:10AM (#39498137)
LLMs don’t come up with new derivative solutions to problems in programming land, from what I’ve seen.

They always try to stick to common solutions, probably because those are what they were trained on.

Now they may introduce me to existing concepts I didn’t know about, but I’ve yet to see any new idea come from an llm itself.

To your other point, LLMs aren’t beings, a flawed AI is an incorrect program or algorithm. Entertaining, maybe, useful in some contexts, sure.

But they’re not beings. nothing about statistical models are like people. I think it’s dangerous to use such fuzzy conflating language with regards to AI.

Re:Every model learned by gradient descent is approximately ... by vidarh on Sunday February 25, 2024 @09:10AM (#39500962)
Most humans don't come up with new solutions to problems with any regularity.

> nothing about statistical models are like people.

As if we're not behaving like statistical models. Our decision-making is so fuzzy and probabilistic that trying to get us to stick to fixed sets of rules consistently is one of the things humanity spends the most time inventing systems to try to handle, and keep failing at. We don't even know a consistent way of achieving it for extremely basic things. We can't keep ourselves to rules we set ourselves with any consistency.

Re:Every model learned by gradient descent is approximately ... by dartos on Sunday February 25, 2024 @09:43AM (#39501172)
You can imagine human behavior is like a stochastic parrot (the name of your argument,) but that’s, again, a gross oversimplification of the human experience.

I don’t really feel like having the stochastic parrot debate again, but look it up.

We’re more than parrots.

Re:Every model learned by gradient descent is approximately ... by vidarh on Sunday February 25, 2024 @11:09AM (#39501920)
The notion of a stochastic parrot is just a subset of this. It is what I'm arguing against, not for, on the basis that talking about "just" a statistical model is meaningless exactly because something being a statistical model does not imply it is just a stochastical parrot. As such it's a wildly misleading claim to make.

Yes, we're more than parrots, but that does not mean we can be described equally validly as "statistical models". Suggesting something is "just" a statistical model is a statement that is close to semantically void. It tells us near nothing about the computational limits - up or down - of something. Even a very simple system for running a statistical model with a loop around it providing IO can be made Turing complete.

Re:Every model learned by gradient descent is approximately ... by vasilipupkin on Sunday February 25, 2024 @08:59AM (#39500877)
Agreed. But in general if I’m looking for a plumber for a huge skyscraper, am I going to hire a plumber who has never worked on one of them and had only done suburban houses ? Most likely, no, I’ll get a recommendation for a pulunbet with experience working on one of those buildings.
Re:Every model learned by gradient descent is approximately ... by bamboozled on Sunday February 25, 2024 @02:17AM (#39498412)
I reakon if you’re getting paid 200k a year to write YAML it might seem like this but being a plumber is a very 3 dimensional challenge.

Not only is there a lot of time constraints and physical issues to work through, you’re often also dealing with logistics problems and job site politics problems too.

Waiting for parts, working out how to keep things going in the meantime. Equipment failures etc. it’s quite chaotic from experience.

There’s a science but also an art to being a good tradesmen.