I think to make it to the next step, AI will have to have some way of performing rigorous logic integrated on a low level.
Maybe scaling that brown-rat brain will let it emulate an internal logical black box - much like the old adage about a sufficiently large C codebase containing an imperfect Lisp implementation - but I think things will get really cool we figure out how to wire together something like Wolfram Alpha, a programming language, some databases with lots of actual facts (as opposed to encoded/learned ones), and ChatGPT.
It still won't magically use them 100% correctly, but with a bit of smarts you can go a long way!
Meanwhile, Wolfram software has built-in methods to solve a lot of different math problems for which in Python you would either need large (and sometimes quirky) libraries, if those libraries even exist.
That and you need to actually expose python to GPT somehow, and Jupyter is not the worst way I suppose.
* The fact that Jupyter holds on to state means GPT doesn't need to write code from scratch for every step of the process.
* GPT can easily read back through the workbook to review errors or output from computations. GPT actually tries to correct errors even. Especially if it knows how to identify them.
To be sure, this is not magic. Consider it more like a tool with limited intelligence; but which can be controlled using natural language.
(Meanwhile, Anthropic allows Claude to run js with react, which is nice but seems less flexible in practice. I'm not sure Claude reads back.)
I don't know if the hardware can be scaled up.
I'm pretty skeptical of the scaling hypothesis, but I also think there is a huge amount of efficiency improvement runway left to go.
I think it's more likely that the return to further scaling will become net negative at some point, and then the efficiency gains will no longer be focused on doing more with more but rather doing the same amount with less.
But it's definitely an unknown at this point, from my perspective. I may be very wrong about that.
Whether those approaches can scale enough to achieve that is relevant to the question, whether the bottleneck is in hardware or software.
I wager that some scrappy resource constrained startup or research institute will find a way to produce results that are similar to those generated by these ever massive LLM projects only at a fraction of the cost. And I think they’ll do that by pruning the shit out of the model. You don’t need to waste model space on ancient Roman history or the entire canon for the marvel cinematic universe on a model designed to refactor code. You need a model that is fluent in English and “code”.
I think the future will be tightly focused models that can run on inexpensive hardware. And unlike today where only the richest companies on the planet can afford training, anybody with enough inclination will be able to train them. (And you can go on a huge tangent why such a thing is absolutely crucial to a free society)
I dunno. My point is, there is little incentive for these huge companies to “think small”. They have virtually unlimited budgets and so all operate under the idea that more is better. That isn’t gonna be “the answer”… they are all gonna get instantly blindsided by some group who does more with significantly less. These small scrappy models and the institutes and companies behind them will eventually replace the old guard. It’s a tale as old as time.
Six million is a start but this tech won’t truly be democratized until it costs $1000.
Obviously I’m being a little cheeky but my real point is… the idea that this technology is in the control of massive technology companies is dystopian as fuck. Where is the RMS of the LLM space? Who is shouting from every rooftop how dangerous it is to grant so much power and control over information to a handful of massive tech companies, all whom have long histories of caving into various government demands. It’s scary as fuck.
Human intelligence improved dramatically after we improved our ability to extract nutrients from food via cooking
https://www.scientificamerican.com/article/food-for-thought-...
We can put a lot more power flux through an AI than a human body can live through; both because computers can run hot enough to cook us, and because they can be physically distributed in ways that we can't survive.
That doesn't mean there's no constraint, it's just that the extent to which there is a constraint, the constraint is way, way above what humans can consume directly.
Also, electricity is much cheaper than humans. To give a worked example, consider that the UN poverty threshold* is about US$2.15/day in 2022 money, or just under 9¢/hour. My first Google search result for "average cost of electricity in the usa" says "16.54 cents per kWh", which means the UN poverty threshold human lives on a price equivalent ~= just under 542 watts of average American electricity.
The actual power consumption of a human is 2000-2500 kcal/day ~= 96.85-121.1 watts ~= about a fifth of that. In certain narrow domains, AI already makes human labour uneconomic… though fortunately for the ongoing payment of bills, it's currently only that combination of good-and-cheap in narrow domains, not generally.
* I use this standard so nobody suggests outsourcing somewhere cheaper.
The average brown rat may use only 60 kcal per day, but the maximum firing rate of biological neurons is about 100-1000 Hz rather than the A100 clock speed of about 1.5 GHz*, so the silicon gets through the same data set something like 1.5e6-1.5e7 times faster than a rat could.
Scaling up to account for the speed difference, the rat starts looking comparable to a 9e7 - 9e8 kcal/day, or 4.4 to 44 megawatts, computer.
* and the transistors within the A100 are themselves much faster, because clock speed is ~ how long it takes for all chained transistors to flip in the most complex single-clock-cycle operation
Also I'm not totally confident about my comparison because I don't know how wide the data path is, how many different simultaneous inputs a rat or a transformer learns from
Only a small part of that 60kcal is used for learning, and for that same 60 kcal you get an actual physical being that is able to procreate, eat, do things and fend for and maintain itself.
Also you cannot compare neuron firing rates with clockspeed. Afaik each neuron in a ml-model can have code that takes several clock cycles to complete.
Also an neuron in ml is just a weighted value, a biological neuron does much more than that. For example neurons communicate using neuro transmitters as well as using voltage potentials. The actual date rate of biological neurons is therfore much higher and complex.
Basically your analogy is false because your napkin-math basically forgets that the rat is an actual biological rat and not something as neatly defined as a computer chip
The conclusion does not follow from the premise. The observed maximum rate of the inter-neuron communication is important, the mechanism is not.
> Also you cannot compare neuron firing rates with clockspeed. Afaik each neuron in a ml-model can have code that takes several clock cycles to complete.
Depends how you're doing it.
Jupyter notebook? Python in general? Sure.
A100s etc., not so much — those are specialist systems designed for this task:
"""1024 dense FP16/FP32 FMA operations per clock""" - https://images.nvidia.com/aem-dam/en-zz/Solutions/data-cente...
"FMA" meaning "fused multiply-add". It's the unit that matters for synapse-equivalents.
(Even that doesn't mean they're perfect fits: IMO a "perfect fit" would likely be using transistors as analog rather than digital elements, and then you get to run them at the native transistor speed of ~100 GHz or so and don't worry too much about how many bits you need to represent the now-analog weights and biases, but that's one of those things which is easy to say from a comfortable armchair and very hard to turn into silicon).
> Basically your analogy is false because your napkin-math basically forgets that the rat is an actual biological rat and not something as neatly defined as a computer chip
Any of those biological functions that don't correspond to intelligence, make the comparison more extreme in favour of the computer.
This is, after all, a question of their mere intelligence, not how well LLMs (or indeed any AI) do or don't function as von Neumann replicators, which is where things like "procreate, eat, do things and fend for and maintain itself" would actually matter.
Have you thought about stepping back from all of this for a few days and notice that you are wasting your time with these arguments? It doesn't matter how fast you can calculate a dot product or evaluate an activation function if the weights in question do not change.
NNs as of right now are the equivalent of a brain scan. You can simulate how that brain scan would answer a question, but the moment you close the Q and A session, you will have to start from scratch. Making higher resolution brain scans may help you get more precise answers to more questions, but it will never change the questions that it can answer after you have made the brain scan.
Num fecisti?
> It doesn't matter how fast you can calculate a dot product or evaluate an activation function if the weights in question do not change.
That's a deliberate choice, not a fundamental requirement.
Models get frozen in order to become a product someone can put a version number on and ship, not because they must be, as demonstrated both by fine-tuning and by the initial training process — both of which update the weights.
> NNs as of right now are the equivalent of a brain scan.
First: see above.
Second: even if it were, so what? Look at the context I'm replying to, this is about energy efficiency — and applies just fine even when calculated for training the whole thing from scratch.
To put it another way: how long would it take a mouse to read 13 trillion tokens?
The energy cost of silicon vs. biology is lower than people realise, because people read the power consumption without considering that the speed of silicon is much higher: at the lowest level, the speed of silicon computation literally — not metaphorically, really literally — outpaces biological computation by the same magnitude to which jogging outpaces continental drift.
Neurons do so much more than a single math operation. A single cell can act as an intelligent little animal on its own, they are nothing like a neural network "neuron".
And note that all neurons act in parallel, so they are billions times more parallel than GPU's even if the operations would be the same.
> A single cell can act as an intelligent little animal on its own, they are nothing like a neural network "neuron".
Unless any of those things contribute to human intelligence, they do not matter in this context.
Cool, sure. Interesting, yes. But only important to exactly the degree to which any of that makes the human they're in smarter or dumber.
To the extent they're independently intelligent, they're the homunculi in Searle's Chinese Room.
> And note that all neurons act in parallel, so they are billions times more parallel than GPU's even if the operations would be the same.
Order of ten million times faster on a linear basis while still a thousand parallel operations.
SpiNNaker needs 100kWh to simulate one billion neurons. So the rat wins in terms of energy efficiency.
> and they tend to be significantly more efficient
Surely you noticed that this claim is false, just from your own next line saying it needing 100 kW (not "kWh" but I assume that's auto-corrupt) for a mere billion?
Even accounting for how neuron != synapse — one weight is closer to a single synapse; a brown rat has 200e6 neurons and about 450e9 synapses — the stated 100 kW for SpiNNaker is enough to easily drive simpler perceptron-type models of that scale, much faster than "real time".
I think current LLMs may scale the same way and become very powerful, even if not as energy-efficient as an animal's brain.
In practice, we humans, when we have a technology that is good enough to be generally useful, tend to adopt it as it is. We scale it to fit our needs and perfect it while retaining the original architecture.
This is what happened with cars. Once we had the thermal engine, a battery capable of starting the engine, and tires, the whole industry called it "done" and simply kept this technology despite its shortcomings. The industry invested heavily to scale and mass-produce things that work and people want.
I think LLMs and modern AI are incredibly amazing and useful tools, but even with the top SOA models today it becomes clearer to me the more I use them that they are fundamentally lacking crucial components of what average people consider "intelligence". I'm using quotes deliberately because the debate about "what is intelligence" feels like it can go in circles endlessly - I'd just say that the core concept of what we consider understanding, especially as it applies to creating and exploring novel concepts that aren't just a mashup of previous training examples, appears to be sorely missing from LLMs.
There is no modern AI system that can go into your house and find a piece of cheese.
The whole notion that modern AI is somehow "intelligent", yet can't tell me where the dishwasher is in my house is hilarious. My 3 year old son can tell me where the dishwasher is. A well trained dog could do so.
It's the result of a nerdy definition of "intelligence" which excludes anything to do with common sense, street smarts, emotional intelligence, or creativity (last one might be debatable but I've found it extremely difficult to prompt AI to write amazingly unique and creative stories reliably)
The AI systems need bodies to actually learn these things.
Or if we're comparing to a three year old, also find the dishwasher?
Closest I'm aware of is something by Boston Dynamics or Tesla, but neither would be as simple as asking it- wheres the dishwasher in my home?
And then if we compare it to a ten year old, find the woodstove in my home, tell me the temperature, and adjust the air intake appropriately.
And so on.
I'm not saying it's impossible. I'm saying there's no AI system that has this physical intelligence yet, because the robot technology isn't well developed/integrated yet.
For AI to be something more than a nerd it needs a body and I'm aware there are people working on it. Ironically, not the people claiming to be in search of AGI.
"Neurons, synapses, signals, chemicals" are downstream of that.
Or just saying, when facing the apocalypse, read a bible?
There is an essential idea of what makes something a dishwasher that LLM's will never be able to grasp no matter how many models you throw at them. They would have to fundamentally understand that what they are "seeing" is an electronic appliance connected to the plumbing that washes dishes. The sound of a running dishwasher, the heat you feel when you open one, and the wet, clean dishes is also part of that understanding.
Can you tell this is a dishwasher? https://www.amazon.com.au/Countertop-Dishwasher-Automatic-Ve...
Can you tell this is a drawing of a glass? https://www.deviantart.com/januarysnow13/art/Wine-Glass-Hype...
Can you tell this is a toy? https://www.amazon.com.au/Theo-Klein-Miele-Washing-Machine/d...
But that won't be the same model that writes bad poetry or tries to autocomplete your next line of code. Or control the legs of a robot to move towards the dishwasher while holding a dirty plate. And each has a fair bit of manual tuning and preprocessing based on its function which may simply not be applicable to other areas even with scale. The best performing models aren't just taking in unstructured untyped data.
Even the most flexible models are only tackling a small slice of what "intelligence" is.
https://sktchd.com/column/comics-disassembled-ten-things-of-...
"The AI systems need bodies to actually learn these things."
I never said this was impossible to achieve.
They can describe it. But do they actually know? Have they experienced it?
This is my point. Nerds keep dismissing physicality and experience.
If your argument is a brain in a jar will be generally intelligent, I think that's pretty clearly wrong.
There are areas of utility here. Things need not be able to do all actions to be useful.
[0] The classic reference: https://en.wikipedia.org/wiki/Principia_Mathematica -- over 1,000 pages, Betrand Russell
[1] https://cmartinez.web.wesleyan.edu/documents/FP.pdf -- a bit more modern, relying on other mathematics under the hood (like DRY reduces the base count), 11 pages
[3] Some reasonable review https://blog.plover.com/math/PM.html
I teach multiple things online and in person... language like that seems like a great to lose a student. I'd quit as a student, it's so condescending sounding. It's only lucky because you get to flex ur knowledge!(jk, pushing it I know lol but i can def see it being taken that way)
Keep in mind I know you're just having fun.
I actually really like the message for 1 in 10,000. As a social outsider for much of my life, it helped me to learn that the way people dismissed my questions about common (to them) topics was more about their empathy and less about me.
But, these sorts of things are difficult to communicate via text media, so we thus persist.
On a side note my couple of times I thought I was treating someone to some great knowledge they should already know I'm pretty sure I came across as condescending. Not bc they didn't know it - i always aim to be super polite - just being young, stupid, and bad at communicating, heh.
And had Russell failed to prove that 1 + 1 = 2 in his system, it would not have cast one jot of doubt on the fact that 1 + 1 = 2. It would only have pointed to the inadequacy of the Principia.
Perhaps your own internal definition of intelligence simply deviates significantly from the common, "median" definition.
Here is what we know: The Pile web scrape is 800GB. 20 years of human experience at 1kB/sec is 600GB. Maybe 1kB/sec is bad estimate. Maybe sensory input is more valuable than written text. You can convince me. But next challenge, some 10^15 seconds of currently existing youtube video, that's 2 million years of audiovisual experience, or 10^9GB at the same 1kB/sec.
Another commenter also mentioned sensory input when talking about the brown rat. As someone who is constantly fascinated at the brains ability to reason/process stuff before I'm even conscious of it, I feel this Stat is Underrated. I'm taking in and monitoring like 15 sensations of touch at all time. Something entering my visual field coming towards me can be deflected in half a second all while still understanding the rest of my surroundings, and where it might be safe to deflect an object. The brain is constantly calculating depth perception and stereo location on every image and sound we hear - also with the ability to screen out the junk or alter our perception accurately(knowing the correct color of items regardless of diff in color temp).
I do concede that's a heck of a lot of video data. It does have similar issues to what I said(lacks touch, often no real stereo location, good greenscreen might convince an AI of something a person intuitively knows is impossible) but the scale alone certainly adds a lot. That could potentially make up for what I see as a hugely overlooked thing as far as stimulus. I am monitoring and adjusting like, hundreds of parameters a second subconsciously. Like everything in my visual field. I don't think it can be quantified accurately how many things we consciously and subconsciously process, but I have the feeling it's a staggering amount.
And that the only reason we think AIs can just be brains in jars is because many of the people developing them consider themselves as simply brains in jars.
> Do they know what a hot shower feels like? They can describe it. But do they actually know? Have they experienced it
Is directly a knowledge argument
Also, it is an argument against physicalism, which I have no interest in debating. While it's tangentially related, my point is not for/against physicalism.
My argument is about modern AI and it's ability to learn. If we put touch sensors, eyes, nose, a mechanism to collect physical data (legs) and even sex organs on an AI system, then it is more generally intelligent than before. It will have learned in a better fashion what a hot shower feels like and will be smarter for it.
I really disagree. Your entire point is about physicalism. If physicalism is true than an AI does not necessarily learn in a better fashion what a hot shower feels like by being embodied. In a physicalist world it is conceivable to experience that synthetically.
Or would you say that you cannot judge the intellegence of someone by reading their books / exchanging emails with them?
My dad is Deaf and doesn't write well, but he can build a beautiful house.
I do wonder if the most energy-efficient way to scale up AI models is by implementing them in organic substrates.
I mean we have dogs. We really like them. For ages, they did lots of useful work for us. They aren’t that much smarter than rats, right? They are better aligned and have a more useful shape. But it isn’t obvious (to me at least) that the rats’ problem is insufficient brainpower.
I mean dogs came with us to the Americas, and even to Australia. Both the Norse and the Inuit took dogs with them to Greenland.
This doesn’t solve the unpredictability problem.
But we haven't solved it for human beings either.
Human brains are unpredictable. Look around you.
While individuals do trust some of the people they know, in the aggregate society doesn't seem to trust human beings to behave reliably.
It's possible, though I don't know for sure, that we're going to need systems and processes to cope with the unpredictability of AI systems.
As it was mentioned by others, we've had thousands of years to better understand how humans can fail. LLMs are black boxes and it never ceases to amaze me how they can fail in such unpredictable ways. Take the following for examples.
Here GPT-4o mini is asked to calculate 2+3+5
https://beta.gitsense.com/?chat=8707acda-e6d4-4f69-9c09-2cff...
It gets the answer correct, but if you ask it to verify its own answer
https://beta.gitsense.com/?chat=6d8af370-1ae6-4a36-961d-2902...
it says the response was wrong, and contradicts itself. Now if you ask it to compare all the responses
https://beta.gitsense.com/?chat=1c162c40-47ea-419d-af7a-a30a...
it correctly identifies that GPT-4o mini was incorrect.
It is this unpredictable nature that makes LLM insanely powerful and scary.
Note: The chat on the beta site doesn't work.
Humans are EXTREMELY unpredictable. Humans only become slightly more predictable and producers of slightly more quality outputs with insane levels of bureaucracy and layers upon layers upon layers of humans to smooth it out.
To boot, the production of this mediocre code is very very very slow compared to LLMs. LLMs also have no feelings, egos, and are literally tunable and directible to produce better outcomes without hurting people in the process (again, something that is very difficult to avoid without the inclusion of, yep, more humans more layers, more protocol etc.)
Even with all of this mass of human grist, in my opinion, the output of purely human intellects is, on average, very bad. Very bad in terms of quality of output and very bad in terms of outcomes for the humans involved in this machine.
Currently the AI stuff kind of sucks because you have to be a giant corp to train a model. But maybe in a decade, users will be able to train their own models or at least fine-tune on basic cellphone and laptop (not dgpu) chips.
It shouldn't be too surprising that anti-establishment folks are more concerned with trillion-dollar companies subsuming and profiting from the work of independent artists, writers, developers, etc., than with individual people taking IP owned by multimillion/billion-dollar companies. Especially when many of the companies in the latter group are infamous for passing only a tiny portion of the money charged onto the people doing the actual creative work.
Tech still acts like it's the scrappy underdog, the computer in the broom cupboard where "the net" is a third space separate from reality, nerds and punks writing 16-bit games.
That ceased to be materially true around twenty years ago now. Once Facebook and smart phones arrived, computing touched every aspect of peoples' lives. When tech is all-pervasive, the internal logic and culture of tech isn't sufficient to describe or understand what matters.
Winning by what metric?
It's also orthogonal to the current corporate dystopia which is using monopoly power to enclose the value of individual work from the other end - precisely by inserting itself into the process of physical distribution.
None of this matters if you have a true abundance economy, but we don't. Pretending we do for purely selfish reasons - "I want this, and I don't see why I should pay the creator for it" - is no different to all the other ways that employers stiff their employees.
I don't mean it's analogous, I mean it's exactly the same entitled mindset which is having such a catastrophic effect on everything at the moment.
I mean, you don't have to look any further than the (justified) lack of sympathy to dockworkers just a few months ago: https://news.ycombinator.com/item?id=41704618
The solution is not, and never has been, to shack up with the capital-c Capitalists in defense of copyright. It's to push for a system where having your "work" automated away is a relief, not a death sentence.
I would engage with those people you're stereotyping rather than gossiping in a comments section, I suspect you will find their ideologies quite consistent once you tease out the details.
Remember Napster? Like how rebellious was that shit? Those times are what a true social upsetting tech looks like.
You cannot even import a video into OpenAI’s Sora without agreeing to a four (five?) checkbox terms & conditions screen. These LLM’s come out of the box neutered by corporate lawyers and various other safety weenies.
This shit isn’t real until there are mainsteam media articles expressing outrage because some “dangerous group of dark web hackers finished training a model at home that very high school student on the planet can use to cheat on their homework” or something like that. Basically it ain’t real until it actually challenges The Man. That isn’t happening until this tech is able to be trained and inferenced from home computers.
I think the training process constitutes commercial use.
great if AI accelerates its destruction (even if it's through lobbying to our mafia-style protect-the-richest-company governments)
Unfortunately this is not the way it's developing. It's more like: are you a normal person without deep pockets? Download a movie with Bittorrent and get a steep fine. Are you a company with hundreds of millions? Download half the copyrighted material on the internet, it's fine.
We are increasingly shifting to a society where the rules only don't apply when you have capital. To some extend, this has always been true, but the scale is changing.
Is the result of an llm an accurate copy or more of an inspiration? What is the standard we use on humans?
Can we code that determination into a system that when a piece of content is close enough to be a copyrighted work, prevents the llm from generating it?
If we were dogs surely we would say that humans were quite skillful, impressively so even, in pattern matching, abstract thought, language, etc. but are hopelessly dumb at predicting past presence via smell, a crow would similarly judge us on our inability to orient our selves, and probably wouldn’t understand our language and thus completely miss our language abilities. We do the same when we judge the intelligence of non-human animals or systems.
So the reason for why no other animal is close to us in intelligence is very simple actually, it is because of the way we define intelligence.
Just to clarify one point: I don't think intelligence is exclusive to humans. I only think that there's a big discrepency that cannot be explained with neuron counts oor the volume of the brain etc. which makes the argument of more hardware and more data will create AGI.
My main problem with the notion of generalized intelligence (in philosophy; I have tons of problems with it in psychology) is it turns out to be rather arbitrary what counts towards general intelligence. Abstract thought and project planning seems to an essential component, but we have no idea how abstract thought and project planning goes on in non-human systems. In nature we have to look at the results and infer what the goals were with the behavior. No doubt we are missing a ton of intelligent behavior among several animals—maybe even pants and fungi—just because we don’t fully understand the goals of the organism.
That said though, I think our understanding of the natural world is pretty unparalleled by other species, and using this knowledge we have produced some very impressive intelligent behavior which no other species is capable of. But I have a hard time believing that humans are uniquely capable of this understanding nor of applying this understanding. For examples, elephants have shown they are capable of inter-generational knowledge and culture. I don’t know if elephants had access to the same instruments as we, that they would be able to pass this knowledge down generations on build up on them.
As I learned it, artificial neural networks were modeled after a simple model for the brain. The early (successful) models were almost all reinforcement models, which is also one of the most successful model for animal (including human) learning.
Is your point that the capabilities of these models have grown such that 'merely' calling it a neural network doesn't fit the capabilities?
Or is your point that these models are called neural networks even though biological neural networks are much more complex and so we should use a different term to differentiate the simulated from the biological ?
This kind of comparison isn't a new idea. I think Hans Moravec[a] was the first to start making these kinds of machine-to-organic-brain comparisons, back in the 1990's, using "millions of instructions per second" (MIPS) and "megabytes of storage" as his units.
You can read Moravec's reasoning and predictions here:
https://www.jetpress.org/volume1/moravec.pdf
---
That's a different take than "human level is this many mips and megabytes", i.e. his claims are about artificial intelligence, not about biological intelligence.
The machine learning seems to be modeled after the action potential part of neural communication. But biological neurons can communicate also in different ways, i.e. neuro transmitters. Afaik this isn't modeled in the current ml-models at all (neither do we have a good idea how/why that stuff works). So ultimately it's pretty likely that a ml with a billion parameters does not perform the same as an organic brain with a billion synapses
Your Moravec article is only looking at what's necessary for computers to have the processing power of animal brains. But you've been up and down this thread arguing that equivalent processing power could be sufficient for a computer to achieve the intelligence of an animal. Necessary vs sufficient is big distinction.
Given their current scale, I don't think we can judge whether current AI systems "represent a dead end" -- or not.
That's why it's almost certain that a biological brain with a billion synapses outperforms a model with a billion parameters.
> the comparison is not apples-to-apples, because each synapse is much more complex than a single parameter in a weight matrix.
... but calling vector-entries in a tensor flow process "neurons" is at best a very loose analogy while comparing LLM "neuron numbers" to animals and humans is flat-out nonsense.
the same foundation that makes the binary model of computation so reliable is what also makes it unsuitable to solving complex problems with any level of autonomy
in order to reach autonomy and handle complexity, the computational model foundation must accept errors
because the real world is not binary
Uhh, yes we do.
I mean sure, we don't know everything, but we know one thing which is very important and which isn't under debate by anyone who knows how current AI works: current AI response quality cannot surpass the quality of its inputs (which include both training data and code assumptions).
> The number of neuron-neuron connections in current AI systems is still tiny compared to the human brain.
And it's become abundantly clear that this isn't the important difference between current AI and the human brain for two reasons: 1) there are large scale structural differences which contain implicit, inherited input data which goes beyond neuron quantity, and 2) as I said before, we cannot surpass the quality of input data, and current training data sets clearly do not contain all the input data one would need to train a human brain anyway.
It's true we don't know exactly what would happen if we scaled up a current-model AI to human brain size, but we do know that it would not produce a human brain level of intelligence. The input datasets we have simply do not contain a human level of intelligence.
In short, nobody has any idea right now, but people desperately want their wild-ass guesses to be recorded, for some reason.
I don't think that's totally true, and anyways it depends on what kind of scaling you are talking about.
1) As far as training set (& corresponding model + compute) scaling goes - it seems we do know the answer since there are leaks from multiple sources that training set scaling performance gains are plateauing. No doubt you can keep generating more data for specialized verticals, or keep feeding video data for domain-specific gains, but for general text-based intelligence existing training sets ("the internet", probably plus many books) must have pretty decent coverage. Compare to a human: would a college graduate reading one more set of encyclopedias make them significantly smarter or more capable ?
2) The new type of scaling is not training set scaling, but instead run-time compute scaling, as done by models such as OpenAI's GPT-o1 and o3. What is being done here is basically adding something similar to tree search on top of the model's output. Roughly: for each of top 10 predicted tokens, predict top 10 continuation tokens, then for each of those predict top 10, etc - so for a depth 3 tree we've already generated - scaled compute/cost by - 1000 tokens (for depth 4 search it'd be 10,000 x compute/cost, etc). The system then evaluates each branch of the tree according to some metric and returns the best one. OpenAI have indicated linear performance gains for exponential compute/cost increase, which you could interpret as linear performance gains for each additional step of tree depth (3 tokens vs 4 tokens, etc).
Edit: Note that the unit of depth may be (probably is) "reasoning step" rather than single token, but OpenAI have not shared any details.
Now, we don't KNOW what would happen if type 2) compute/cost scaling was done by some HUGE factor, but it's the nature of exponentials that it can't be taken too far, even assuming there is aggressive pruning of non-promising branches. Regardless of the time/cost feasibility of taking this type of scaling too far, there's the question of what the benefit would be... Basically you are just trying to squeeze the best reasoning performance you can out of the model by evaluating many different combinatorial reasoning paths ... but ultimately limited by the constituent reasoning steps that were present in the training set. How well this works for a given type of reasoning/planning problem depends on how well a solution to that problem can be decomposed into steps that the model is capable of generating. For things well represented in the training set, where there is no "impedance mismatch" between different reasoning steps (e.g. in a uniform domain like math) it may work well, but in others may well result in "reasoning hallucination" where a predicted reasoning step is illogical/invalid. My guess would be that for problems where o3 already works well, there may well be limited additional gains if you are willing to spend 10x, 100x, 1000x more for deeper search. For problems where o3 doesn't provide much/any benefit, I'd guess that deeper search typically isn't going to help.
Scaling experiments are routinely performed (the results are not encouraging). To say we know nothing about this is wrong.
It’s interesting how a relatively small # of synapses can do all abstract reasoning when free from those concerns.
Take the pre-frontal cortex, leave the rest.
For your implication to be plausible, you either need to deny that consciousnes plays a role in reasonability of thinking (making you a physicalist reductionist) or you need to posit that a neural network can have consciousness (some sort of mystical functionalism).
As both of these alternatives imply some heavy metaphysical assumptions and are completely unbased, I'd advise to avoid thinking of neural networks as an analogue of brains with regards to thinking and reasonability. Don't expect they will make more sense with more size. It is and will continue to be mere statistics.
All I'm saying above is that the number of neuron-neuron connections in current AI systems is still tiny, so as of right now, we have no way of knowing in advance what the future capabilities of these AI systems will be if we are able to scale them up by 10x, 100x, 1000x, and more.
Please don't attack a straw-man.
The number of neuron-neuron connections in current AI systems is still tiny compared to the human brain.
The largest AI systems in use today have hundreds of billions of parameters. Nearly all parameters are part of a weight matrix, each parameter quantifying the strength of the connection from an artificial input neuron to an artificial output neuron. The human brain has more than a hundred trillion synapses, each connecting an organic input neuron to an organic output neuron, but the comparison is not apples-to-apples, because each synapse is much more complex than a single parameter in a weight matrix.[a]
Today's largest AI systems have about the same number of neuron-neuron connections as the brain of a brown rat.[a] Judging these AI systems based on their current capabilities is like judging organic brains based on the capabilities of brown rat brains.
What we can say with certainty is that today's AI systems cannot be trusted to be reliable. That's true for highly trained brown rats too.
---
[a] https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n... -- sort in descending order by number of synapses.