News for hackers. Stuff that matters.
Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @08:42AM (#40001483)
This is why driverless cars are still just demos, and why LLMs will never be reliable.

It hard to build something that is yet to be even logically defined.

Hoping that AGI will somehow just "emerge" from an inert box of silicone switches (aka a "computer" as we currently know it) is the stuff of movie fantasy ... or perhaps religion.

Re:Scaling will never get us to AGI by tivert on Thursday April 11, 2024 @08:48AM (#40001529)
> ...is the stuff of movie fantasy ... or perhaps religion.

IMHO, something like "movie fantasy" constitutes the core belief of a great many software engineers and other so-called rational people in the tech space.

Re:Scaling will never get us to AGI by galangalalgol on Thursday April 11, 2024 @08:56AM (#40001592)
Driverless cars are here, without liability issues they would already be widespread. But scaling isn't what got them there. It was tons of labeled data, both from rules based implementations and shadowing human drivers.

The ability to generalize completely outside training data isn't that common among humans IME. That is a high bar. How many of us have done so without at least drawing an analogy to some other experience we have had? Truly unique thinkers aren't that commonplace. I myself could have probably just asked an LLM what I would say in this post and gotten pretty close...

Re:Scaling will never get us to AGI by parpfish on Thursday April 11, 2024 @09:19AM (#40001804)
> But scaling isn't what got them there. It was tons of labeled data, both from rules based implementations and shadowing human drivers.

Isn't the second half of that saying that scaling up the dataset was what got them where they are?

When I hear about "scaling up" models, I think there are two parts of it: 1) use the same architecture, but bigger 2) make the training data bigger to make use of all those new parameters.

So when I hear about something that's not scaling, it would require some sort of fundamental change to architecture or algorithms.

Re:Scaling will never get us to AGI by staticman2 on Thursday April 11, 2024 @10:17AM (#40002414)
> Driverless cars are here, without liability issues they would already be widespread.

In the sense that if you were allowed to run over anyone without legal consequences lots of tech enthusiasts would sign up for this experience?

Re:Scaling will never get us to AGI by rchaud on Thursday April 11, 2024 @09:12AM (#40001718)
> Driverless cars are here, without liability issues they would already be widespread

In other words, they're in a lab somewhere undergoing perpetual test runs so they don't kill people, so not generally available.

It is like saying that email was available in the '80s, you just couldn't message anybody outside of the university network.

Re:Scaling will never get us to AGI by naasking on Thursday April 11, 2024 @09:20AM (#40001827)
You can order driverless taxis right now in some cities:

https://www.axios.com/2023/08/29/cities-testing-self-driving...

Re:Scaling will never get us to AGI by loire280 on Thursday April 11, 2024 @09:46AM (#40002093)
However, they require constant human monitoring and interventions from those humans every few miles to work properly - and it's not clear that the need for those interventions will be solved with more training data.

https://www.theverge.com/23948708/cruise-robotaxi-suspension...

Re:Scaling will never get us to AGI by tivert on Thursday April 11, 2024 @11:10AM (#40003035)
> However, they require constant human monitoring and interventions from those humans every few miles to work properly

It's a little-known fact that AI means "Actually, Indians."

https://boingboing.net/2024/04/03/amazons-ai-powered-just-wa...

Re:Scaling will never get us to AGI by MatthiasPortzel on Thursday April 11, 2024 @09:43AM (#40002064)
That article is discussing Cruise. Since it was published, a Cruise car dragged a pedestrian 20 ft, it came out that they need remote intervention every few miles, the CEO resigned, and they stopped doing rides anywhere.

Waymo remains available to my knowledge.

Re:Scaling will never get us to AGI by minitoar on Thursday April 11, 2024 @09:23AM (#40001854)
No, I think he meant they are literally here. I see one everyday.
Re:Scaling will never get us to AGI by the8472 on Thursday April 11, 2024 @08:58AM (#40001605)
Evolution is an incredibly dumb, very parallel search over long timescales and that managed to turn blobs of carbon soup into brains because that's a useful (but not necessary) tool for its primary optimization goal. But that doesn't make carbon soup magical. There's no fundamental physics that privileges information-processing on carbon atoms. We don't have the time-scales of evolution, but we can optimize so much harder on a single goal than it can.

So I don't see how it's a movie fantasy any more than bottling up stars is (well, ICBMs do deliver bottled sunshine... uh, the analogy is going too far here). Anyway, the point is that while brains and intelligence are complicated systems there isn't anything at all known that says it's fundamentally impossible to replicate their functionality or something in the general category. And scaling will be a necessary but perhaps not sufficient component of that, just because they're going to be complex systems.

Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @11:12AM (#40003048)
But that doesn't make carbon soup magical.

The equivalent of a human brain just "emerging" from inert silicon switches without any real understanding of how or why --- that's PFM (Pure Friggin' Magic). There is no logical reason to believe it is even possible or practical --- yet people still believe.

It's the modern day version of alchemy --- trying to create a fantastical result from a chemical reaction before it was understood that nuclear physics is the mechanism required. And even with this understanding, we have yet to succeed at turning lead into gold in any practical way.

Re:Scaling will never get us to AGI by Windchaser on Friday April 12, 2024 @12:10PM (#40014501)
"It's the modern day version of alchemy --- trying to create a fantastical result from a chemical reaction before it was understood that nuclear physics is the mechanism required. And even with this understanding, we have yet to succeed at turning lead into gold in any practical way."

Well, sure. But then, we never observed nature turning lead to gold in any cheap way, whereas we do observe nature running complex intelligence on relatively cheap hardware (our brains).

There's a pretty big difference between trying to do something never Nature did (alchemy), versus trying to replicate in a kinda-parallel kind of way something we've observed Nature doing (intelligence).

Re:Scaling will never get us to AGI by chrsw on Thursday April 11, 2024 @10:36AM (#40002667)
It seems evolution developed intelligence as a way to for organisms to react and move through 3D space. An advantage was gained by organisms who can "understand" and predict, but biology just reused the same hardware for location, moving and modeling the 3D word for more abstract processes like thinking and pattern recognition.

So evolution came up with solutions for surviving on planet Earth, which isn't necessarily the same as general problem solving, even though there are significant overlaps. Just the $.02 of a layperson.

Re:Scaling will never get us to AGI by RohMin on Thursday April 11, 2024 @10:27AM (#40002550)
This is actually a great perspective to have. The idea that there is no fundamental law of physics that should prevent us from replicating or exceeding the functionality of the human brain through artificial means makes this a question of when, not if.
Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @01:42PM (#40004779)
The idea that there is no fundamental law of physics that should prevent us ...

Maybe ... someday.

But right now, we don't even fully understand the physics that makes a human brain work. Every time someone starts investigating, they uncover surprising new complexity.

https://theness.com/neurologicablog/is-the-brain-analog-or-d...

Re:Scaling will never get us to AGI by the8472 on Thursday April 11, 2024 @03:19PM (#40005781)
The understanding is not a prerequisite to build one though. Evolution didn't need it. And all the functionality that already emerges from ANNs doesn't need it either. You mostly need to understand the optimizer, not the result. Of course understanding the result is desirable to steer the optimization process better. But if all you care for is getting any result, even potentially undesirable ones, then building bigger optimizers appears to work.
Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @04:17PM (#40006387)
Evolution didn't need it.

Yes, it only took a few billion years of trial and error. And it didn't manage to do it with inanimate objects either.

We know that the human brain is organic, mostly analog and much more complex than a digital computer.

Re:Scaling will never get us to AGI by the8472 on Friday April 12, 2024 @07:46AM (#40011471)
That's an argument about complexity. And "organic" simply means based on molecules with a carbon backbone. They're versastile, but that does not give them any known information processing advantage over silicon.

If analog processing were relevant (it most likely isn't) then Artifical Neural Networks could also be implemented on analog silicon circuits.

As for the complexity, well, the systems are getting more complex. Straight lines on log charts. That's what scaling is about. Getting there, to that complexity.

So I'm just not seeing any knockout argument from you. We can't predict the future with certainty. But the factors you present do not appear to be fundamental blockers on the possibility. You're pointing at the lack of an existence proof... which is always the case before a protoype.

It sounds like you're basically abandoning forward-thinking and will only acknowledge AGI when it hits you over the head. A sign of intelligence is also the ability to plan for an unseen future.

Re:Scaling will never get us to AGI by tlb on Friday April 12, 2024 @08:00AM (#40011567)
Can you clarify what you mean by "no information processing advantage?" Does increased speed or memory capacity provide such an advantage? Or would you also claim that 2024 silicon has no advantage over 1994 silicon despite several orders of magnitude more speed & memory?
Re:Scaling will never get us to AGI by the8472 on Friday April 12, 2024 @08:18AM (#40011729)
Since we're looking towards the future and asking whether there is an advantage of the basic materials (organic vs. inoraganic) I'm talking about the physical limits of those substrates, whether there's anything special about them that allows one to process information in a way that the other can't. I.e. how many bits can be stored in a cubic centimeter of nano-structured silicon compared to a cube of carbon, oxygen, hydrogen and other organic chemistry arranged into (for example) neurons. How many TFlop/s can be fit into such a cube in principle etc. etc. Or if there are some other physical processes relevant to information processing that make carbon special. Those fundamental limits have not changed over time, the physics are the same. All that changes are how much use engineering makes of those possibilities.
Re:Scaling will never get us to AGI by Anotheroneagain on Thursday April 11, 2024 @09:18AM (#40001789)
The question is if other animals even do have the kind of intelligence that most people call AGI. It seems that this kind of adaptive intelligence may be unique to mammals (and perhaps a few other outliers, like the octopus), with the vast majority of life being limited to inborn reflexes, immitating their conspecifics, and trial and error.
Re:Scaling will never get us to AGI by bionhoward on Thursday April 11, 2024 @09:36AM (#40001994)
It’s not a hope to expect machines to match or exceed the human level of intelligence, because we demonstrate human level intelligent machines by virtue of existence, and the physical limits of computation far exceed those of our specific biology.

AKA, we know it’s possible for atomic systems to be as intelligent as we are, because we are. We suspect the limit is far beyond us because computers have dramatically faster processing, essentially perfect memory systems, etc

Re:Scaling will never get us to AGI by myrmidon on Thursday April 11, 2024 @09:26AM (#40001882)
Do you agree that AGI can emerge from a bunch of haphazardly connected Neurons, using about 700MiB of procedurally generated initialization data (human DNA) and employing 1-2 decades of low-bandwidth, low-power (~25W) training?

You might even agree that not a single directed thought went into the design of that whole system.

Having established that, it seems laughable to me that it would be an "impossible fantasy" to replicate this with purpose-built silicon.

Sure, our current whole-system understanding could still be drastically improved, and the sheer scale of the reference implementation (human connectome) is still intimidating even compared to our most advanced chips, but I have absolutely zero doubt that AGI is just a matter of time (and continuous gradual progress).

If a lack of understanding did not stop nature, why should it stop us? :P

To me, all the arguments against AGI appear unconvincing, motivated by religion/faith, or based on definitional sophistry.

But I'm very open to have that view changed...

Re:Scaling will never get us to AGI by spangry on Thursday April 11, 2024 @09:36AM (#40002000)
Well put. I’ve wonder if some of those on the other side of the argument are actually, perhaps on some subconscious level, proponents of mind-body dualism: https://en.wikipedia.org/wiki/Mind%E2%80%93body_dualism
Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @12:46PM (#40004187)
To me, all the arguments against AGI appear unconvincing, motivated by religion/faith, or based on definitional sophistry.

Religion is belief in the unknown without reason or logic.

Do you know of any inanimate object that is truly "intelligent"?

Without ever knowing or seeing a single, real working example, you "believe" and have "faith" that it is possible. By so doing, you are practicing religion --- not science.

Re:Scaling will never get us to AGI by the8472 on Thursday April 11, 2024 @03:38PM (#40005971)
That's like complaining about artificial, non-organic flight being a fantasy before the wright brothers. Which would not have been entirely wrong, but hardly a knockout argument as we now know with certainty.

There is a vast difference between "engineering challenges have not yet been proven moot by a working prototype" and "precluded by physics or mathematics".

The (in)animate distinction is more akin to net-positive fusion in a man-made vessel vs. in a gravity well rather than obeys the 2nd law of thermodynamics vs. perpetuum mobile.

Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @04:09PM (#40006295)
That's like complaining about artificial, non-organic flight being a fantasy before the wright brothers.

Nope.

Before the Wright brothers, we knew it was scientifically possible to suspend objects heaver than air is an air current. For example, kites and balloons.

The only examples we have of "intelligence" are organic in nature.

Since we have no examples to the contrary; for all we know, "life" and "intelligence" could be somehow inter-related.

And we don't currently fully understand or know how to engineer either one. Yet people have "faith" --- just like the alchemists.

Re:Scaling will never get us to AGI by the8472 on Friday April 12, 2024 @07:36AM (#40011400)
> The only examples we have of "intelligence" are organic in nature.

High-abstraction category-words such as "intelligence" are not fundamental properties of nature. They're made by man. Which means we currently happen to define intelligence in a way that we primarily observe in organic objects. So there's some circular reasoning in your argument. If we look at all the more fundamental building blocks such as information-processing, memory, adaptive algorithms, manipulating environments then we know all of them already are implementable in non-organic systems.

It is not a blind belief decoupled from reality. It is an argument based on the observation that the building-blocks we know about are there and they have not been put together yet due to complexity. There also is the observation that the "putting together" process has been following a trajectory that results in more and more capabilities (chess, go, partial information games, simulated environments, vision, language, art, programming, ...), i.e. there's extrapolation based on things that are already observable. Unless you can point at some lower-level piece that is only available in organic systems. Or why only organic systems should be capable of composing the pieces into a larger whole. I am not aware of any such limiting factor.

Just like "suspending" objects in air was possible and self-propelled machines were possible, even if not self-propelled flight had not yet been done at that time.

Re:Scaling will never get us to AGI by myrmidon on Thursday April 11, 2024 @07:22PM (#40007863)
This is very nonconvincing to me, because you dance around the core of your own argument, refusing to state it plainly: Why would it be necessary for everything intelligent to consist of animated matter? You don't even offer a plausible hypothesis.

To me the null hypothesis is that non-organic intelligence is possible --at worst (!!)-- simply by emulating exactly what an intelligent organic entity does (but it seems obvious to me that this approach is likely to be wasteful and inefficient).

Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @07:36PM (#40007955)
Why would it be necessary for everything intelligent to consist of animated matter?

It may not be necessary --- but at this point in time, we really don't know enough to state this as a fact.

What we do know is that our only working examples of "intelligence" are all organic and mostly analog --- not digital. Suggesting that real "intelligence" is possible without "life" --- is unsupported by any evidence and is a pure leap of faith at this time.

Re:Scaling will never get us to AGI by myrmidon on Thursday April 11, 2024 @08:06PM (#40008103)
First, "It doesn't exist already" is an extremely poor argument against the technical feasibility of really anything :P

> Suggesting that real "intelligence" is possible without "life" --- is unsupported by any evidence and is a pure leap of faith at this time.

Strongly disagree on this; If you accept that brains effect intelligence by selfinteraction accoring to physical laws, then "computability" directly and inevitably follows. And that implies artificial brains are feasible...

Re:Scaling will never get us to AGI by jqpabc123 on Friday April 12, 2024 @11:15AM (#40013881)
And that implies artificial brains are feasible...

Given enough time, money, effort and energy, anything is *feasible* --- even converting lead into gold.

Expecting it to just *emerge* somehow on it's own without any real design or plan or understanding of the scope involved --- that goes way beyond *feasible* or *practical* and heads straight for *magical thinking*.

Re:Scaling will never get us to AGI by Windchaser on Friday April 12, 2024 @12:17PM (#40014577)
Is anyone expecting it to "just emerge somehow"?

This seems strawmanny. There has been an absolute ton of research into figuring out what kind of design/scope is necessary. More planning and research is happening all the time, and architecture is specifically being changed with a mind towards specific new functionalities. It's not haphazard; it's planned. Yes, there is some room for 'emergence' or learning, but that doesn't mean there isn't also a lot of structure and planning.

The picture you present of NN research is so far off from what I see actually being done in this field that I'm a little boggled. Are you sure you're in touch with what's actually happening in the field?

Re:Scaling will never get us to AGI by dragonwriter on Thursday April 11, 2024 @08:09PM (#40008116)
> Do you know of any inanimate object that is truly "intelligent"?

So, your argument is that by making something intelligent we would necessarily also make it animate?

That's reasonable, but doesn't impact the plausibility of AGI. “Artificial” is not “inanimate”.

Re:Scaling will never get us to AGI by lelanthran on Thursday April 11, 2024 @05:15PM (#40006896)
> To me, all the arguments against AGI appear unconvincing, motivated by religion/faith, or based on definitional sophistry.

Correction. I'm atheist. I don't see AGI for the future. Same reasons as the poster who responded to you below: evolution may not have had a design, but we know it works well on animated creatures.

We have yet to see any examples of inanimate objects exhibiting any sign of intelligence, or even instinct.

What we do know is that, even after billions of years of evolution, not a single rock has evolved to exhibit a sign of intelligence.

You're saying that with an intelligent hand directing the process, we can do better. I understand the argument, and I concede that it is a reasonable argument to make, I'm just unconvinced by it.

Re:Scaling will never get us to AGI by myrmidon on Thursday April 11, 2024 @07:45PM (#40007997)
> We have yet to see any examples of inanimate objects exhibiting any sign of intelligence, or even instinct.

Can you give me a clear, minimal definition for both "intelligence" and "instinct"?

Because to me, instinct seems to be already achieved by existing inanimate control systems for decades now, and "decent understanding of natural human language" (as achieved by todays LLMs) is more than enough to count as intelligent for me.

If we just started simulating neurons in a brain exactly, what would prevent us from achieving inorganic intelligence in your view?

Re:Scaling will never get us to AGI by lelanthran on Friday April 12, 2024 @02:32AM (#40009944)
I don't really want to get involved in a discussion on where the fine dividing line is between intelligence and non-intelligence. We are fine determining intelligence at the extremes, but not in the large shades of grey in-between.

Most people (say, 999999 out of every 1000000) will consider a rock unintelligent and Stephen hawking to be intelligent. You can't call them wrong because "they cannot define intelligence".

> If we just started simulating neurons in a brain exactly, what would prevent us from achieving inorganic intelligence in your view?

Nothing. But the word "If" is doing all the heavy lifting in that argument. I mean, we aren't doing that at the moment, are we? It's not clear that we might ever discover exactly how the collection of neurons we call a brain "exactly works".

IOW, if the human brain was simpler, we'd be too simple to understand it: this may already be the case!

And the evidence for "we may not figure out how the brain works exactly enough to clone the mechanism it uses" is a lot larger than "we might figure out how the brain works well enough to clone the mechanism it uses."

This is why I remain unconvinced. Even though I think your position is a reasonable one to hold, the opposite position is, IMHO, just as reasonable. It's got nothing to do with religion or superstition.

Re:Scaling will never get us to AGI by Windchaser on Friday April 12, 2024 @12:24PM (#40014666)
"What we do know is that, even after billions of years of evolution, not a single rock has evolved to exhibit a sign of intelligence."

Is it even appropriate to apply "after billions of years of evolution" to rocks? Rocks don't evolve. Evolution can't act on them.

And, we aren't generally looking to "evolve" AI, so it's kinda a moot point there, too.

What is it about animation that, to you, is important for intelligence? Presumably it's the training data + agency, but.. are these not possible without physical-world "animation"?

But also: how come AI can't be animated?

Re:Scaling will never get us to AGI by iwsk on Thursday April 11, 2024 @08:55AM (#40001578)
General intelligence somehow just emerged from dirt and water + @.
Re:Scaling will never get us to AGI by jqpabc123 on Thursday April 11, 2024 @11:54AM (#40003588)
And the Sun just somehow emerged from a cloud of hydrogen gas.

The obvious flaw in this line of thinking (as related to AGI) is that we/humans can create/engineer the same without any real understanding of the underlying mechanisms involved.