>We don't know what "intelligence" means, but we know it isn't matrix multiplication, or brute force algorithms, otherwise gears could be called intelligent.

We know no such thing. The simplicity of the basic operations do not necessarily constrain the complexity of the whole system composed of those operations. We compose simple operations into complex units all the time.

> We compose simple operations into complex units all the time.

Through a process called thinking, too.

Which itself is a complex process composed from simple operations.

Twas the joke

If you built a complex series of gears that took input, revolved through different sets of millions of gears, and produced meaningful output, I would consider that a form of intelligence.

Agreed. My understanding of NN is that more than matrix multiplication, it’s that it’s a general purpose solver. You could write the same thing yourself, it would just take you ages.

So with unlimited budget and time, can you write something complex enough to seem intelligent? I think so. Is what you wrote actually intelligent? No idea, and I think that’s more philosophy than I’m interested in.

General purpose solving functions will only get better with time and already solve more than we can write solvers for by hand. I don’t suspect there’s a limit here, assuming we can keep improving in ways to scale its compute and scale the function goals.

You are basically describing an automaton, some of them were very elaborate and able to write with a pen on paper. People probably misinterpreted it for intelligence, basically thing have improved, but not changed.

Whether you build it with gears, transistors on silicon wafer, or biological neurons, it doesn't matter. If complex enough arrangement of enough neurons can give rise to intelligence, then so can enough transistors or enough gears.

"It makes the whole world afraid of what is in fact just matrix multiplications" is such a reductionist view that doesn't really capture the reality of AI taking people's jobs and reshaping the economy. That's like saying electricity is just "electrons moving through a wire"

You and the comment you're referring to are both making good comments, but may not be talking apples to apples.

I believe the post you're replying to is just saying that matrix multiplications (as useful as they are), aren't going to become Skynet.

Your post is pointing out that various AI techniques are replacing loads of jobs for folks that still need to make ends meet, but likely don't have the skill sets to magically become a web developer overnight. As a result, AI is pretty dangerous like many disruptions throughout history. Only in the past, there was usually still plenty of need for labor.

That's a deep thought, it puts the discussion in a great perspective for further thinking. Thanks!

These matrix multiplications won't become Skynet, but that has nothing to do with the fact that they are matrix multiplications.

The better we get at understanding how various cognitive functions that once seemed mysterious can be accomplished by matrix multiplications, the easier it becomes to eventually create Skynet.

Also consuming enormous amounts of power and water

Look bombs are just an accelerated oxidization. Like rusting but fast. What's the big deal. Rusting is natural.

For the most part electrons don't move through the wire, that would be very inefficient.

“Just a bunch of matrix multiplications” is also a bit odd because lots of jobs have been automated out of existence by tools way less complicated than matrix multiplications.

The weird thing, I think, about these matrix multiplications, seems to be that they might be coming for the jobs of people who are generally in the same field that invented them (programmers) and also they might be coming for the jobs of reporters, creatives, and hot-take authors. People with bigger platforms than factory workers.

>We don't know what "intelligence" means, but we know it isn't matrix multiplication

I didn't, how do you know? And to be honest many LLM seems to show intelligence to some degree, at least when they solve complex and novel problems I ask them in a random language and using N library, that's feel pretty intelligent to me and if it's not than also humans are not.

I do see where you are coming from, but perhaps it is good that people think of it as proper AI (with all the inherent concerns) so that we enact rules and regulations before there is a runaway ML arms race that actually does give us AGI. Will we be ready then otherwise?

Great response. I will take some time to think about it, thanks!

> but we know it isn't matrix multiplication

Neuroscientist here: You're right!

At least in the neuro that we know today, most [0] of the communication between neurons is done in the Fourier domain. In that, it's the frequency of firing events that matters, not that a neuron fired at all.

[0] by no means is it exclusive. The brain is really complicated and there are edge cases all over the place.

The fourier domain is just a linear transform of the time domain, i.e. just a matrix multiply away!

Could you expand please? I'm connecting the two via convolution but can't intuitively grasp the link yet.

See “Laplace transform”

https://en.m.wikipedia.org/wiki/Laplace_transform

The fourier transform converts between time domain and frequency domain. The fourier transform is linear, which means it can be implemented as multiplying your time domain signal by a matrix of complex fourier coefficients. Usually we dont do it this way because the fft is faster than matrix multplication, but the result is the same.

Wow, could you point me to other resources to read about this?

There has been a long discussion in the neuroscience community how neuronal firing encodes information. The two extremes of the proposed solutions are temporal coding and rate coding, but there are intermediate positions such as held by the parent poster (and myself, and nowadays most other people that care about this niche).

Wikipedia is decent-ish on the topic: https://en.wikipedia.org/wiki/Neural_coding

Wulfram Gerstner's book has a nice discussion of the theory and limitations of different proposed coding schemes, which might interest the technical audience here. In particular, I think the parent poster alluded to the ideas presented in the last section here: https://neuronaldynamics.epfl.ch/online/Ch7.S6.html

Really and truly this "discussion" should have been put to rest a long time ago. Neurons encode information with the temporal precision needed to encode that information. Since most objects are somewhat permanent, the temporal precision can be very low, yielding effectively a rate code. For tasks for which the (relative) timing is critical, e.g. the encoding of auditory sequences, the temporal precision of the neural code is high, yielding effectively a temporal code. Some problems are neither here nor there, in which case the neuron will encode as much information as possible at a low rate / temporal resolution, and then encode diminishing amounts of information at successively higher temporal resolutions (the Fourier code alluded to by the parent poster). Higher temporal precision typically requires more control, larger circuits, and hence more energy expenditure.

Bill Bialek's group had a paper that quantified the amount of information present in a visual neuron of the fly at different temporal resolutions (/frequencies) as early as 2008: https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

Is there a certain way to encode typical "continuos" information like a certain tilt of a line in an image?

I ran into this over ten years ago in an experiment on temporary plasticity of sensory input and I didn't find a quick answer before I had to leave the topic (ok, it left me)

Thanks for sharing that book link, btw.

> We don't know what "intelligence" means, but we know it isn't matrix multiplication

What makes you believe “intelligence” can’t emerge from lots of matrix multiplications? Unless you believe in some more mystical explanation, human intelligence is just electrochemical processes not that unlike computers.

That "it isn't matrix multiplication" argument is completely equivalent to "no computer can do it", and to "nobody can ever understand it". And is practically equivalent to "you need a soul to have intelligence".

The same applies to "brute force algorithms" and "otherwise gears could do it".

It's very likely that the current crop of LLMs do the wrong set of matrix multiplications. (If you ask me, it's a certainty.) But that doesn't change the fact that matrix multiplications can do anything.

> The problem is the "intelligence" word causes more harm than value.

By this measure "learning" could be inaccurate too. Does a human learn if they just commit something to memory? Programming is chalk full of people who know the code but haven't "learned" to program. Intrinsically we all know there's something deeper to learning than just memorizing or retaining to memory.

I think GP has a point. Marketing terms are to the benefit of the user to be relatable, whether accurate or not. We can, and to my knowledge, historically do retain the stories of what the correlation of a marketing term versus its actual technology is.

I think they can be right but also missing the point. Inaccurate but relatable statements can miscommunicate risk. If a marketer calls something “self driving intelligence” when an engineer prefers the more precise term “driver assistance software” it can lead to bad consequences when the consumer doesn’t understand the nuance.

sadly to most of the world, if you can do matrix math, you're intelligent.

> I understand the importance of selling. But selling shouldn't be confused with deceiving consumers

Woo boy, someone hasn't taken marketing 101 in uni.

First, "artificial intelligence" is the name of the field of study that tries to build systems that act intelligently, not a term for referring to the produced systems. Even if some outcome isn't (or wasn't) yet particularly intelligent, the name of the field is entirely appropriate since that's the goal which it is striving to approach.

Second, the general consensus of what "intelligence" means in the AI field is approximately the Legg&Hutter "Intelligence measures an agent’s ability to achieve goals in a wide range of environments." from https://arxiv.org/abs/0706.3639 .

It's worth noting that solely focuses on the capabilities of the system and intentionally completely disregards how it's built. Gears cannot be called intelligent iff it is impossible to build an system that acts intelligently solely from gears, and if it turns out that it is possible to build a gear arrangement that soundly demonstrates intelligent behavior, then those gears definitely could be called intelligent.

And it's also a clearly nonbinary definition - i.e., it's not whether something is or isn't intelligent, but a scale to measure how intelligent it is; and a observation that a particular system (or human!) demonstrates poor intelligence doesn't prevent us from discussing that it's more intelligent than something or someone who is even worse.

“puffery” and forward looking statements do not constitute claims of material fact. I.e., they don’t meet the legal definition of fraud.

Nothing discussed in this article is related to the concerns people have with the forefront of AI research. No one is afraid that the first computer that can convince >50% of the world that its sentient will be a SaaS app, or at least I hope not!