Penrose's point is that natural selection seems to have capitalized on a specific physical feature which can do something better than any computer can -- this he finds quite remarkable, and he is expressing skepticism that, if you just leave a genetic algorithm for a while with an arbitrary weighting function, that it will develop structures which start to reach towards such logic puzzles with curiosity and form proofs of them.
That we have done this is not testament to the anthropic principle nearly as much as it is testament to the fact that, among all the animals, we seem to have most cleanly decoupled our logical intuitions and applied them to our problems.
I'm a little confused about how the anthropic principle explains this fallacy (and it is a fallacy to be sure). I guess you're referring to how natural selection (and all the other sorts of selection that occur regularly) lead to the existence of "appropriate" organisms and features.
But I think knowing some of the actual details in how evolution works at all the different levels (molecular bio, epigenetics, competition/cooperation, behavior, etc, etc, etc) serves as an even better explanation. Here's a 25 lecture course from Stanford that explains just that: http://www.youtube.com/watch?v=NNnIGh9g6fA
The anthropic principle simply invalidates this mystery: it is not special, it simply happened, and because it happened, we exist to observe it.
It doesn't care to explain evolution in any biological or scientific way; it only seeks to remove all doubt that it was able to happen at all.
Strictly speaking, a fallacy is a defect in the structure of an argument, i.e. in the logic that connects a premise to a conclusion. What you and the parent have identified is simply a false premise ("Things at least seem to organize themselves somewhat better than they 'ought' to"[1]). A false premise may result in the wrong conclusion, but an argument can be wrong without being fallacious.
This distinction isn't so much relevant here, but it comes in handy when you want to pinpoint sources of disagreement in a discussion. Lobbing the f-word around (even when deserved) just makes people defensive.
1. Of course the argument used to derive this premise might be fallacious, but that argument isn't given.
1. A mistaken belief, esp. one based on unsound argument.
2. A failure in reasoning that renders an argument invalid.
We were using it in the first sense.Let's assume we are but one of an infinite path of possibilities for slightly different starting conditions of a universe. When you get to this magnitude of combinations, it is no longer sufficient to compare individual combinations -- rather it is infinite sets that make the most sense to compare -- classes of possible universes in a sense. You are comparing cardinalities, not members of the different sets themselves.
And if you think about all of the possibilities, it is almost certain (borrowing the mathematical usage of that term) that our universe is unlike almost all combinations, and to a significant degree. I could be wrong here, but it is certainly something that could be proven (or disproven) by an expert mathematician.
So, when you say we won a lottery, I think it is instead more like you threw a dart at a real number line between 0 and 1 and the dart landed exactly on 0.4. It's not impossible, but the probability of that happening is still 0. (Those two statements are actually not incompatible. There's a good question on the math stack exchange about it).
It's not based on what I want to believe, or even something from self-selection bias. It is something that can be calculated and either proven true or false, probably with set theory of some kind.
I generally use the anthropic principle in the context of evolution and our existence on this planet—of all planets in the universe—versus our existence in this universe. You could certainly take it to that extreme, but it's a much deeper venture and we know so little concrete data about it.
How many universes are there? Is it an infinite set, as some suggest? We don't know. Do they differ in their stability or cosmological constants? We don't know. How many universes have there ever been? We don't know. How much time has ever existed? We don't know. Does time even exist outside the 4-dimensional space of our universe? We don't know that, either.
But we do have a very good theory about how we came about, as sentient lifeforms, on this specific planet around this specific sun in this specific universe. That's quite a lot, actually.
If you can prove anything mathematically about the probability of our universe's existence by random chance as opposed to some outside factor, then book your flight to Stockholm. My most humble hypothesis is that you will require more data to run those numbers; data that we do not yet have. You could make some guesses, but that's all they'll be.
In the end, it boils down to the age old endless question: originally it was "how were we created?"; then it was, "how was the Earth created?", then "how was the solar system created?", followed by galaxy, space, the universe, whatever's holding the universe, ad infinitum. At the ever-expanding edge of our knowledge, "God" seems as good a word as any to describe that mystery.
I'm not saying no to the fact that it could have randomly landed there, but given that the probability is 0, I am saying that it is possible there is something else we have not discovered that provides a more probable explanation than chance.
*A different type of analogy is the old spontaneous generation theory for maggots in meat. For a long time, people thought they just randomly appeared. Upon closer investigation, it turns out there were other, more deterministic processes going on.
If I flip a coin 5 times in a row, HHHHH is just as likely as any other combination, but humans have the tendency to think that is weird or special.
Anyway...
If I understand your hangup correctly, it seems you are mistakenly thinking that all imaginable possible realities should have been equally likely (as positions on a random dart board would be). Of course this is not true; a reality where snakes developed internal organs that function as internal combustion engines which they use to power propellers is very unlikely indeed. The reason of course is that this reality did not arise by picking one randomly from a bag of all possible realities, but rather developed over time through random mutations that were selected for by natural selection.
Now, of the possible realities random mutation and natural selection could have created, this world could be said to be equally likely. There is not some sort of "future purpose" as Penrose says, that is being driven towards.
Our universe is also only specified to finite precision; we don't know whether physics is ultimately continuous or discrete, but regardless, we can only measure anything to finitely many digits. So whatever the probability of our universe may be, it is at least greater than zero.
Of course we already know about many processes that support the formation and existence of life in this already existing universe. Stellar evolution for example, is a rather healthy branch of scientific inquiry. There are not "mysterious" things at play, unless Penrose and yourself are simply intending to imply that there are still things which we do not fully understand (which is obvious, and that is a very confusing way of stating something so obvious and universally recognized).
People upvote for agreement, it only makes sense that they downvote for disagreement as well. I'm fairly certain that PG has said that he is ok with that before.
You can be pretty sure that you will always be downvoted for complaining about downvotes though, even by people that may agree with you.
Evolution as a process is entirely reasonable when you look at the details if the logic.
"To my way of thinking there is still something mysterious about evolution, with its apparent 'groping' towards some future purpose. Things at least seem to organize themselves somewhat better than they 'ought' to, just on the basis of blind-chance evolution and natural selection."
This is a common fallacy about evolution, and is explained beautifully by the anthropic principle, or in other words, the innate selection bias of our existence. We've self-selected for our own awareness of our circumstance and existence. Things are not organizing better than they "ought" to, they've just happened to organize to a sufficient point that we exist and perceive this process and say things about it like the above quote.
It is in the same way that someone who wins the lottery must think themselves exceedingly lucky that they, of all the millions of people participating, have won. They must think there is something mysterious about this, that things turned out somewhat better than they 'ought' to, just on the basis of blind chance.
Yet, what is the probability that some person, of the entire pool of people in the world, wins the lottery? One. It has necessarily happened by the nature of the lottery.
We as a species have won this lottery, by the mere nature of our sentience. We should not think it mysterious or unusual in any way. However, we are lucky in the sense that we are here; we are special in that we can perceive and understand. As long as we understand the fact that there is no "should" in evolution, this is a perfectly fine thought. it just happened, and on this planet, it produced something able to understand itself. As Carl Sagan said, "We are a way for the cosmos to know itself." Certainly there is much metaphysical and philosophical consequence to our existence, but scientifically and probabilistically speaking, it makes perfect sense.
Consequently, I believe it may be much more difficult to reach true AI than some have postulated.