"If you wish to create a brain in 50 MB of code, you must first implement the universe"?
It is, of course, conceivable that the brain contains some kind of computational processes that can be implemented/emulated much more quickly on a biochemical substrate than in doped semiconductors — some kind of quantum computation or something. However, all the evidence we have at the moment points in the opposite direction, and there's no particular reason to believe that the universe is designed to give biochemical systems such an advantage.
The CPU may be physical, but software on it runs on the abstraction which is the CPU's instruction set. The underlying physical nature of the processor is irrelevant.
He was talking about having one machine emulate another - I was pointing out that that means your i7 emulating all of physics.
Very nice try. (I mean that sincerely.) However, you must also have a proof about the complexity of the emulation and the encoding of programs to emulate. Without that proof - which basically means you are no longer talking about two arbitrary machines but rather about two which are deeply related - the mere possibility of emulation tells you nothing other than that one machine is of equal or greater computational power than the other.
[Edit: in context - sure, our fancy big super computer from IBM can emulate the zygote (let's say) but.. what's that gonna take? Trillions of lines of code? 10 lines?]
And, maybe I need another cup of coffee but I am actually having trouble parsing the syntax of your comment about Omega.
If we're talking about Turing machines, all of the machines are of equal computational power, and are also able to emulate each other.
If we're talking about real machines, the take away is that moving from one machine to the other doesn't require much code, but does require that the other machine be powerful enough. Thus leading to Ray Kurzweil's argument that the real challenge is getting a powerful enough machine, and not in the amount of code needed to run on that machine. (He further argues that Moore's law will provide us with that machine.)
And, maybe I need another cup of coffee but I am actually having trouble parsing the syntax of your comment about Omega.
Basically I was saying that if you offer some facility to translate programs from machine A to machine B, only a very small portion of the programs on machine B will turn out to be programs that were translated from machine A. Therefore the probability of programs halting on machine A tells us nothing about what the majority of programs on machine B are likely to do, and therefore tells us very little about the probability of a random program on machine B halting.
Does that make more sense?
We aren't necessarily talking about Turing machines but let's suppose that we are. We're talking about machines powerful enough to compute anything that any Turing machine can compute. You are correct that any one of these machines can emulate any of the others.
What's relevant to Kurzweil's argument is two-fold:
If I tell you about Turing-class machine A and give you Turing-class machine B, we know right away that you can write an A emulator that runs on B. We know that because they are both Turing class. What we don't know is how big that emulator program is. It might take a trillion lines of code just to write the emulator.
Next, I have told you about machine A but now I also give you a 50 Megabyte program (called "GENOME") that runs on machine A. Now you must write your emulator, encode GENOME for your emulator, and run it on B. We know you can do that since these are both Turing class machines. What we do not know is how many bits it will take to encode GENOME. It could be 50 Megabytes. Or it could be 50 Terabytes. You just can't tell without proving things about the specific details of machines A and B and your emulator.
Kurzweil makes an argument analogous to "Hey, GENOME only takes 50MB on machine A. There's no way it takes 50TB on machine B. Information theory at work, man! Use your head!" Problem is, "information theory" says no such thing. Actually, it says "If that's all we've been told, hell it could easily take 50TB on machine B. It could take 5 bytes. Who knows? There's not enough said about machines A and B or the program GENOME to tell!"
Basically I was saying that if you offer some facility to translate programs from machine A to machine B, only a very small portion of the programs on machine B will turn out to be programs that were translated from machine A. Therefore the probability of programs halting on machine A tells us nothing about what the majority of programs on machine B are likely to do, and therefore tells us very little about the probability of a random program on machine B halting.
Yes. You are correctly observing that Omega is not really a constant - it is a class of constants. For machine A there is Omega_A. For machine B, Omega_B. That is why Chaitin lectures about Omega but only after several reminders that it's really "Omega_X for some specific machine X -- but I'm mostly just going to say Omega and let X be understood."
So, now, think that through with Kurzweil goggles on. In Kurzweil's world, a program of 50MB on one machine can't be 50TB on the next machine -- that's just silly, right? Well, if he were correct then, in general, Omega_A and Omega_B, for any machines A and B, would have to be somewhat close in value. We know that Omega_A and Omega_B don't have to be close (other than that they both come from the same (0,1) interval). By contradiction we conclude that we don't live in the world Kurzweil is describing.
If your emulator requires substantially different space to encode the raw description of the program than the original, then the emulator sucks. So if GENOME takes 50 MB to encode in one place, it should take 50 MB to encode in another. Certainly in any emulator written for real computers, it wouldn't take more than that.
But we're lacking the emulator. That's important because nobody knows how big or little that will be. The point that the biologist made boils down to saying that the emulator is an incredibly complex thing that has to encode an environment whose complexity is much, much more than that of the genome. So one flaw in that line of reasoning from Kurzweil is that we have no known upper limit on the size of the emulator itself. The biologist's argument basically comes down to, "The emulator itself needs to be much, much bigger than GENOME, and we have no idea how to write it."
(Note that I'm not arguing that Kurzweil is correct, but merely that your arguments against him were not correct.)
Yes. You are correctly observing that Omega is not really a constant - it is a class of constants. For machine A there is Omega_A. For machine B, Omega_B. That is why Chaitin lectures about Omega but only after several reminders that it's really "Omega_X for some specific machine X -- but I'm mostly just going to say Omega and let X be understood." So, now, think that through with Kurzweil goggles on. In Kurzweil's world, a program of 50MB on one machine can't be 50TB on the next machine -- that's just silly, right? Well, if he were correct then, in general, Omega_A and Omega_B, for any machines A and B, would have to be somewhat close in value. We know that Omega_A and Omega_B don't have to be close (other than that they both come from the same (0,1) interval). By contradiction we conclude that we don't live in the world Kurzweil is describing.
Not so fast. As I pointed out, even if the emulator was just a ridiculously small 50 bytes, then Omega_A would tell us very little about Omega_B because only about 1 program in 10120 on machine B would be programs that are emulated from machine A. You know nothing about the halting probability of the rest, and so Omega_A says very little about Omega_B.
(Obviously that 50 bytes is ridiculously small. But in http://www.cs.auckland.ac.nz/~chaitin/lisp.html Chaitin has an interesting example of a Lisp variant that can run a theorem prover against itself with only 410 bytes of overhead. So with pairs of machines, you can get away with surprisingly little.)
The complexity of a Turing machine is a function of the number of states that it can encode, in programming languages we call that 'expressiveness'. You can write a smaller program in a language that is more expressive. A Turing machine that allows more states is going to require a shorter tape (fewer cells) to do its work but will be more complex.
The argument I think that Kurzweil makes with his 50 MB figure is that once you take out all the redundancy you've taken out the expressiveness factor. Whatever emulator you use to interpret that 50 MB you'll end up with some 3D structure (if you simulate the embryology properly) that will resemble a brain.
The part where the analogy goes wrong according to me is by postulating a machine that this stuff should run on. The 'emulator' is nothing but the endless repetition of ribosomes threading RNA in order to produce more polypeptide chains which then fold up in order to become proteins.
Your 'emulator' in this case is a physics program, and not even a very complicated one, albeit with an absolutely astronomical data set for any size organism.
In order to avoid having to deal with that data set we try to achieve a higher level model of what the brain does, instead of trying to model what it is.
But that does not change the fact that the principle of the emulator (a 'mere' matter of programming) and the code that governs the construction are what gives rise to the brain and through there to intelligence.
So the upper bound of the problem in terms of complexity is the 50 M + whatever you need in terms of physics to (theoretically) run the simulation of the single fertilized ovum from start to finish, or as much as you need to verify that you have indeed succeeded in your simulation goal.
It can't get more complex than that, even if the generated structure is orders of magnitude more complex at first glance than the input data + the program. Think of it as 'output'. Thought is not the output of that program. Thought is the state changes within the produced (or simulated) brain.
The organic soup and all the proteins arising from it are the nuts and bolts the machinery is made of. That doesn't mean that we can't make similar machinery without those particular nuts and bolts.
I hope that all makes sense.
For instance http://www.nature.com/nature/journal/v441/n7092/full/nature0... has an example of RNA-based inheritance that is not in the genome. Also http://en.wikipedia.org/wiki/Genomic_imprinting shows that what genes you have and what genes you express are two different stories entirely.
That said, the effort of transcribing the 50 MB of genomic information into the final program is simply not a problem. And if we wanted to transcribe a much larger genome, the additional effort for that is again bounded.
This is all, of course, ignoring the excellent point that one would never want to actually do this given how slow this emulator would have to be. If we create AI in the next few decades, I can guarantee that it won't be a physics emulation of a human brain.
When you argue that a simple (just a few rules of physics) emulator could, in principle, do the job -- quite right. Unfortunately for your point, Kurzweil is making a much, much stronger claim. For, surely, you agree that that physics emulation isn't going to result in a practical program in 20 years, yes?
The way you are using "upper bound of the problem in terms of complexity" there... I could use your same logic to argue that the design of a brain can be encoded in exactly 1 bit.
The design of a brain, by your logic, contains 1 bit of information. (In particular, that 50 MB of compressed genome data? I'll give you a better compression algorithm for this purpose and it will, indeed, compress a particular human genome down to 1 bit.)
Since all Turing complete machines are functionally equivalent I don't think it is quite that irrelevant.
if the brain can be simulated by any Turing machine at all it figures that a 2-d tape Turing machine can do the same thing.
The question then becomes how big is the total information content of both Turing machine + tape.
The rules of physics are Turing complete, we use them to build computers with and nature uses them to build the physical universe, including computers and brains.
Kurzweil is not making such a strong claim at all I think, he merely argues that there must be some upper boundary to the starting configuration from which you should be able to arrive at a working brain.
And much as I would like to I can't find fault with that other than that the embryotic development is chaotic in nature and that a very small change in input conditions can easily produce nothing of interest on the output side.
The conclusion that we can draw from that is that the brain (and in fact probably every cell in our bodies) must contain a great deal of machinery to make sure that the process stays on track and does not veer off in to chaos and unpredictable outcomes. For the most part, it works, most of the fetuses carried to term are able to think.
> For, surely, you agree that that physics emulation isn't going to result in a practical program in 20 years, yes?
Some people claim even shorter times than that (which I think is highly irresponsible):
http://www.dailymail.co.uk/health/article-1201724/Artificial...
Personally I would put the time required for such a project in the multiple decades, probably 50 years at a minimum, maybe much more assuming that it can be done and that there is no quantum component (as proposed by Penrose, rather weakly) to intelligence.
As for your 1 bit description, you are not taking in to account the information content of the guts of the compression algorithm, and you'd find that algo+compressed data would work out to a nearly constant amount of information.
That 50MB has been created in a way that the size of the algorithm required to compress it and decompress it is relatively small compared to the size of the data.
As soon as you start embedding codebooks inside your algorithm the size of the data will go down but the size of the algorithm goes up correspondingly. Also the algorithm loses its universal character, it's ability to encode multiple data sets with roughly equal characteristics in terms of information content in to roughly equally sized output.
I can't try to help you any further. You have a conceptual bug in your understanding of the relevant math / CS.
Last I'll say is that your emulator doesn't necessarily "suck" - it may be that it is the best that is theoretically possible and still result in exponential growth in emulated program size.
And, you also don't seem to be following a relatively simple argument about Omega.
It's interesting stuff and we started down this thread on an interesting foot... I only mean here to explicitly sign off and note that, in my opinion, you're just not getting a few basic things I'm not too sure how better to present to you (which is not to say I'm normally particularly good at explaining such things -- only that for whatever reason, in this case, it ain't working).
About the emulator, the length of the emulated program is the length of the emulator plus the length of the representation of the program. On standard machine architectures you can represent binary data directly with little overhead. (Somewhere you have to declare the length of the binary data, so there is some overhead.) If you're encoding binary data within a programming language you may have a constant factor of encoding overhead because of your need to avoid reserved characters, etc. This is typically a small constant factor. I'm aware of no practical examples where that overhead is exponential, or even super-linear.
About Omega, I understand the argument that you're trying to make. Your argument is that if Omega_A and Omega_B are to vary, then some programs from A have to go to much longer programs on B. My point is that this is not true, all that is necessary is that the programs from A have to be lost in a much larger sea of valid programs in B. Given that the number of programs of length n varies exponentially in the size of n, there are enough programs that this will happen even if all programs in A go to programs that are just a fixed number of bytes larger in B. Therefore Omega_A tells you very little about Omega_B.
Yes, right there you are demonstrating a deep misunderstanding of what you are replying to or how it relates to Kurzweil's arguments.
On Omega, you say: all that is necessary is that the programs from A have to be lost in a much larger sea of valid programs in B
You left out that we're comparing sets of A and B programs of similar length -- the "sea" is not so large. Actually you kind of acknowledge this but then say bogus things about what it implies.
You are confused or else just b.s.'ing on purpose. It's been fun. We're done, please.
You left out that we're comparing sets of A and B programs of similar length -- the "sea" is not so large. Actually you kind of acknowledge this but then say bogus things about what it implies.
In this case "similar" was "differs by a constant". Thanks to exponential growth, even if said constant is something small like 50 bytes, the pool of programs on machine B would be 2^400 times as big as the pool of programs on machine A. That's about a factor of 10^120.
If you think that a factor 2582249878086908589655919172003011874329705792829223512830659356540647622016841194629645353280137831435903171972747493376 difference is "not so large", then you've got a very different impression of what "large" means than most people do.
Anyways now I think we're going around in circles. Unless you make a point of some substance in your next reply, I'm not going to bother replying. And no, insults do not count as points.
Well, you know, part of the other half of the "source code" in the genome explains how to build a zygote. It doesn't explain how to build a universe, but there's no particular reason to believe that the universe is designed to make zygotes easy to build.
The encoding scheme isn't particularly problematic; it's a constant factor on the efficiency, and a small additive increment to the source code.
Not true! Worst case scenario, one machine can emulate the other machine, running that input. Therefore an upper bound on one machine can be used to derive an upper bound on the other. These upper bounds are not perfect, but they are indicative.
Indicative is good enough for the argument that Kurzweil is attempting. His argument has other flaws, but this is not one of them.
That is why, for example, when Chaitin lectures about Omega he is always mumbling "relative to some choice of turing machine" (at least initially, until it is then understood to apply throughout the talk).
Not so fast. Omega is the probability of a randomly chosen machine halting. Suppose that an upper bound on machine A predicted that for machine B within just 50 bytes of data. Then all of the programs of length n on machine A get translated on machine B to a list of programs that is 2400 times bigger, and the rest have unknown behavior. In this case Omega for machine A says very, very little about Omega for machine B.
Therefore "indicative" is very much not good enough for what Gregory Chaitin is talking about.