His numbers are also based off the difference between humans and chimps for some reason? We can't replicate the intelligence of a chimp on a computer, so why that is a valid basis of comparison is beyond me.

Quite. You would suspect that if we were able to produce an AI with the intelligence of a chimp then a human-level AI would be in reach. The harder step is probably getting to the level of chimp intelligence; you'd have managed to create consciousness and solved the hard problem for a start.

It's a much, much harder step, actually.

https://en.wikipedia.org/wiki/Moravec%27s_paradox

Kind of renders all of his information theoretic calculations moot.

Mind if I go a little off-topic here? This paradox reminds me of some other conclusion I've had, which is the fact that energy requirements of many physical processes are completely counterintuitive. For instance, it probably takes more energy to light a light bulb for a day, than it takes a small processor to run a set of calculations equivalent to a lifetime's worth of arithmetic done by every human being alive today.

>Kind of renders all of his information theoretic calculations moot.

The stuff in the article isn't information theory. Information theory, in the senses of Shannon and Chaitin, is actually very useful for studying cognition, because it tells us about how much we can try to learn, via what methods, from noisy sensory data.

Moravec's paradox is not an experimental result. It's just one guys musings on intelligence. It says a lot more about the misguided things people thought about intelligence in the early days of AI than the actual inherent difficulty of achieving human-level AI.

According to the linked article, several notable researchers were of the same opinion (and one might also say that relativity was just one guy's musings - though admittedly experimental evidence wasn't long in coming.)

I do, however, agree with your second sentence, and I think the paradox is also weakened by the fact that when 'high-level' mental tasks are computerized (chess-playing, for example), it is often through brute-force calculation that is not a model of how humans use their general intelligence to perform the task.

He mentions that later in the post:

> Chimps are remarkable thinkers in their own right. Maybe the key to intelligence lies mostly in the mental abilities (and genetic information) that chimps and humans have in common. If this is correct, then human brains might be just a minor upgrade to chimpanzee brains, at least in terms of the complexity of the underlying principles.

Of course this still kills the argument that it's at most 125MiB in my opinion. Moravec's paradox [1] implies a lot of the actually-hard tasks related to inference and learning are already solved in chimps, because they can see and hear and balance just fine. I think if we actually understood how chimps reasoned, in high-level and low-level terms, then we could easily tweak and improve that process to function beyond human level.

1: https://en.wikipedia.org/wiki/Moravec%27s_paradox

I believe Moravec's paradox is a consequence of the fact that we do not consciously recognise the effort our brains/bodies do in sensorimotor computation. When we add up numbers consciously in our head, using arithmetic, it takes us quite some effort, but in the meanwhile our visual cortex is doing similar operations (unconsciously) millions of times more efficiently.

It's as if the conscious arithmetic runs on some kind of stack of virtualization layers, using millions of neurons to build a mental image of the concept of "the number five", whereas a handful of neurons would suffice to do the actual arithmetic.

I often wonder if those rare people who can do calculations in their head faster than calculators, may have - unconsciously - found a way to unlock the native hardware of their brain, bypassing all of the symbol abstractions required by us muggles. A bit like GPGPU vs CPU for certain algorithms - only orders of magnitude more pronounced.

We can't even replicate the intelligence of a rat or even a cockroach, let alone a chimp.

> A multi-cellular embodied intelligence can acquire information about the patterns of the physical world by growing in it - the code is compact because much of the necessary information can be acquired during embodied growth.

Not sure why you think "embodied growth" is essential here; having a compact code base that can be supplemented by information during maturation obviously has some utility, but why would you think embodied growth is critical to it?

> A disembodied intelligence such as AIs can't rely on external storage and acquisition of information during a cellular growth stage

AIs aren't disembodied. They have physical substrates, just like natural intelligences.

I don't see why something like cellular growth is necessary for "external storage and acquisition of information" during development. Even if it were, however, just as natural multicellular organisms are coordinated colonies of self-reproducing cells, artificial organisms (intelligent or otherwise) could be coordinated colonies of self-reproducing machines.

AIs are not disembodied; they obviously are embodied in physical machines. And there's no reason an AI couldn't acquire information during development,

I think the GP's point is just that there is unaccounted-for information about existing in the physical world encoded into that embodiment. AI will also have embodiment, and there will necessarily be information stored there, but a great deal of operational information will have to be learned and explicitly expressed in the AI itself. To use your example, an artificial organism could be made of coordinated colonies of machines, but cells already have a huge head start in the way they bind to one another, transport materials within and without, respond to various signaling agents, etc. Down to the basic molecular building blocks of the cellular machinery has been selected to (attempt to) optimize operation across all actions that cell might take.

The argument isn't that that embodiment is necessary for the artificial system to learn how to operate in a way that's as capable or better, just that there's a whole lot of information there not accounted for in the genetic code alone.

It also ignores the possibilities (1), that cells and tissues are extremely complex (and thus require a lot of genetic code), while the overall algorithm the cells and tissues perform might still be very simple, and conversely (2), that chimpanzees might be already extremely intelligent and the difference is rather a matter of embodiment (in particular the ability to produce language), and perhaps minor tweaks to neurons, to the cortex and/or other brain regions.

According to the Santiago Theory of Cognition

"Living systems are cognitive systems, and living as a process is a process of cognition. This statement is valid for all organisms, with or without a nervous system." https://en.wikipedia.org/wiki/Santiago_Theory_of_Cognition

If this were a correct interpretation, even Bacteria could be seen as already incredibly intelligent. Going Zero to One (dead matter -> prokaryotes) is a significantly bigger step than going One to Five (prokaryotes -> humans).

Careful, modern prokaryotes also have billions of years of evolution behind them. The first organism where likely inefficient and would not last in our current hyper competitive ecosystems. Arguably, the first organism where really more just free floating proteins without even a cell wall.

yeah agreed. i considered writing proto-prokaryotes.

>> This ignores that genetic code is meaningless without the translation machinery

Not really. The translation machinery is also defined in the DNA. Granted, there is a bootstrapping issue. The chimp argument seemed silly to me. Why not just estimate so fraction of the human genome as defining the brain? In that case 3G base pairs is a bit under one gigabyte of information defining an entire human. Still small in comparison to the quantity of synapses.

The translation machinery is nowhere represented in the DNA. Its the cell soup, that came from mother and her mother before her, back to the primordial soup. Dna is the paper tape; the egg is the computer (or autofac, or self-replicating machine etc). That paper tape has nowhere the instructions for making the egg; only detail for how it operates.

>> The translation machinery is nowhere represented in the DNA.

Why yes, yes it is. There are genes that code the proteins that make up the ribosome for example. The ribosomes in turn interpret the DNA string to make proteins. This leads to a bit of a chicken and egg problem which I mentioned as the issue of bootstrapping. But the DNA does code everything in the cell, or at least defines the machinery needed to make everything.

I think the DNA does contain instructions for making the egg.

For example, if executing the DNA instructions produces a human female, she'll be born with a lifetime supply of eggs, the developmental instructions for which were part of her startup DNA. Those instructions were definitely not part of the "cell soup", as is shown by mitochondrial donation assisted reproduction technology (aka 'three-parent babies').

The egg made the eggs. The DNA said "make an egg". That's the idea. Like a paper tape that you feed into a factory, with one punch, labeled "Man/Mouse". Punch 'Man' and feed it in, and the factory makes a man. So, a man is encoded in one bit? No.

Cells are factories, incredibly complex ones. DNA is a set of punches in a tape.

Yeah, the genetic code alone is like having a bunch of source code with no compiler, no spec to write a compiler, and the language is something you've never seen before and it has no similarities to anything you had seen before.

It's like being giving a bunch of Brainfuck source--if you'd never heard of Brainfuck.

The embodied cognition perspective provides such an interesting counterpoint to the notion that a brain is just a computer in these philosophical debates over AI vs humanity.