I think you are missing the main thrust of the book. He is not trying to convince you that em-based AI is the likely future. He chose the less examined possibility as a starting point, but the main part of the book is to enumerate the predictions that flow from that premise.

This is interesting and worth examining even if the premise is wrong. Sometimes examining a familiar topic in unfamiliar terrain yields new insights.

"False premise" is a logical fallacy, not a virtue.

A false premise isn't a logical fallacy. Arguments can be logically valid yet wrong, and vice versa.

Right, and taking a premise you believe to be false and mapping out its consequences has a long and noble history; e.g., non-Euclidean geometry.

Okay then...if not a logical fallacy, then just a fallacy ("mistaken belief").

The sort of prognostication that would do poorly on a debate team, because its premises are unsound.

You don't have to think that this is the most likely scenario to think this is a very important book. Even if the chance that em AI beats ab initio AI is only 5%, it's a dramatically different and important scenario to consider.

I attended a primer lecture on tbe immune system. Holy hell. At least 200 different types of T cells. At least 200 other types of immune cells. Then there are all their interactions, signaling, locating, memory functions. I honestly dont know if one can ever get a handle on the complexity.

It is not complexity, but diversity. The functions those cells perform are mostly understood as are most mechanisms by which they work. There are many variants of a similar function and mechanism.

That said, immunology is still quite young too, but compared to it neuroscience is not even born.

For a large part we understand how two particles can interact. Simple rules.

Put together 200 of those particles, predictions become very difficult and complex.

Alan Kay's inspiration for OOP and message passing seems to have come from working in microbiology.

Maybe thinking/finding about how biology does its computation will be important for machines that can scale very well?

I think a lot of these predictors underestimate prediction itself. To their credit, there is a lot of emphasis on trends in economies or technologies. That's probably the best tool for predicting, but predicting is hard. I think it's hard to really know how complicated "biology" is, from the current perspective of mystery. Who knows what'll be known decades from now.

I like Elon Musk's perspective as a starting point though. The trends that all these predictions rely on are man made. Technology improves, gets more powerful and more prevalent when smart people work on it hard. It's not a physical force, it's caused by people.

Anyway, if you are expecting narrow AI to do so much, who knows what the relationship between that technology and "true" AI (whatever that means exactly) is. It may not matter much, as you say.

> I like Elon Musk's perspective as a starting point though

what is elon's perspective ?

He talks about the time estimate in the book. He says he understood the need for a time estimate, and so gives the one he thinks is best. But the important thing is that he feels confident it will happen before artificial intelligence that isn't full-brain emulation, whether that's in one century or four centuries.

"confident" is too strong. I think the chance is high enough to make the premise worth considering in this detail.

> On top of the sheer audacious difficulty of creating whole brain emulations, there's the problem of competing with boring old "narrow" AI. Surely an economist of all people should recognize the feedback loop that makes narrow AI profitable first, which attracts more investment, which unlocks more capabilities without biological emulation... how many of today's jobs are going to be left un-done by narrow AI a century from now, even if whole brain emulations are finally practical then?

This is a curious blind spot. With deep learning algorithms making easy work of master Go players, there has to be enormous untapped potential in GPGPU silicon for AI.

He's stated elsewhere he thinks the chances of this scenario are only ~30% IIRC

Based on what, though? The hazard of dealing with Bayesian epistemologists is that they will confidently state percentages that they literally made up.

Based on his own personal belief. Probabilities are a very useful way to precisely state the certainty of beliefs.

It means that if you offered him betting odds that it would happen, of less than 1:2, he would take it. And he would bet against it at odds higher than 1:2. Of course betting on the far future is impractical, because money might not be worth anything if it does happen, but in principle that's how these numbers can be interpreted.

Yes, I know where Bayesian epistemologists get their numbers.

The danger is that they then announce these numbers as if they have any real-world basis, and that they are taken as making a claim about objective reality.

There's no evidence I know that pulling a number out of your backside and running it through Bayes' theorem produces better results than just pulling the number out directly and running with it. Indeed, the real-world track record of LessWrong/MIRI (to bring this back to topical relevance) strongly suggests it's less effective.

He didn't claim it had "real world basis" or "run it through Bayes' theorem". He just said he expects about a 30% chance he is right.

Is that different from any other epistemologists?

Yes, biology is terrifically complex and there exists a near-universal tendency to oversimplify it.

However, the rate at which humans are improving our ability to understand terrifically complex systems is itself accelerating and 100 years is a very long time.

The initial stages of a sigmoid curve resemble exponential growth. I'm pretty sure that the exponential character we've witnessed is the result of miniaturization of the transistor and the industrialization of large third-world economies, both of which are running out of steam. If we don't get some major new breakthroughs, expect to see the other side of that sigmoid soon enough.

We may see another brief exponential curve with the development of carbon nano-computers, likely netting at least an order of magnitude additional computing performance. I have strong doubts about the feasibility of general purpose quantum computing (though it will definitely have niche uses).

If you extrapolate from the last hundred years of progress to the next hundred, it's plausible that there will be at least as much technological progress as these ideas would require.

The problem with such prognostication isn't so much that these things can't happen. It is that unexpected events that will happen first turn the future in a different direction.

There are others who think it an important possibility: http://www.fhi.ox.ac.uk/wp-content/uploads/brain-emulation-r...

I was involved with a research effort to predict certain cell biology features via molecular dynamics simulations between 2003 and 2006. It's what I wrote my thesis on. After school was done and my research group's grant ran out I started writing software for a living instead of looking for another research position.

I still keep up with molecular and biological simulation research efforts because it's intrinsically interesting to me. The models are improving and better results are also possible due to the increase in computing power since 2006. But the progress appears slow compared with AI/ML research over the same period.

Any time covalent bonds are made or broken, quantum effects are at work. Eliminate all the chemistry that involves forming or breaking bonds and you're throwing away most of the chemistry in a living organism. Getting quantum effects right with ad hoc modifications to molecular dynamics models based on classical physics is very difficult; there's a lot of risk of overfitting to what's easy to observe. Full quantum calculations have terrible scaling properties -- O(N^7) for CCSD(T), the most popular high accuracy approximation of QM, where N is proportional to the number of electrons in the system. Even the best, slowest quantum methods (CCSD(T) and even more exotic methods) still have many cases where they fail to reliably reproduce experimental results.

It's possible that you don't need to simulate anything down to the molecular level to reproduce all the biologically interesting features of cells. I don't want to rule it out completely. But the enormous financial incentives to simulate biology in silico (for e.g. accelerating pharmaceutical development), the large research investments made so far, and the limited progress to date tend to make me think that all those tiny and extremely computationally expensive details at the molecular level are biologically relevant.

This has been my main objection to brain inside of a computer thinking. If we could simulate the universe inside of a computer we could do all kinds of neat things, such as drug development in a computer, but it isn't algorithmically possible. The Kurzweillians are just assuming that we'll wave the magic wand and have efficient algorithms for everything such that computer simulation of reality will be trivial. If you have an exponential complexity algorithm that is in the middle of your problem like protein folding for example it will NEVER be practical to simulate in a computer on any non-trivial problem.

I don't want to rule out better algorithms developing very good approximations for what currently appear to be daunting problems like protein folding. But there are a lot of problems like folding out there where even a century of Moore's Law marching on wouldn't be enough to solve large instances with today's algorithms. And Moore's Law looks like it's going to do more shuffling forward than marching forward in the future.

So basically I agree with you: there's no generic march-of-progress that assures a world of Kurzweillian wonders. I want to see more concrete evidence of progress in relevant areas before I start entertaining life after the Singularity. (I expect a future profoundly shaped by many "narrow" AIs rather than AGI precisely because there has been so much concrete progress with the former compared to the latter.)

This brings to my mind an interesting line of thought that I ran across via Peter Watts writing about it in Blindsight [1]. In that link, page down to the "Sentience/Intelligence" header, and read that section for a concise introduction to this line of thought: sentience is separate from intelligence. Looking at what computer science research and industry has accomplished thus far, I'm very open-minded about the idea that sentience (feeling and perceiving) is a separate, older, deeper layer of brain processing from sapience (reasoning and intelligence).

I suspect at a layman's comprehension level that sapience is mostly what we are reproducing in research and industry today, and not sentience, because sentience is far more difficult to plumb. Your explanation of the challenges in quantum methods of simulation and the uncertainty on whether it is even necessary seems to bring up this line of questioning to me. This has interesting ramifications. Like we can build OCR and even put out of work lots of formerly middle-class attorneys who made their living in trial discovery phases. But we have yet to create a second Industrial Revolution with general purpose humanoid-form robots that everyone can own that can clean houses, mow lawns, fold clothes, make beds, wash and chop vegetables, pick strawberries, etc., all supposedly "simple" tasks.

http://www.rifters.com/real/Blindsight.htm#Notes

I think this is right on. People use rational thought to evaluate decisions, but they use emotions to make decisions. If you take away a person's ability to feel emotions, they become paralyzed with indecision. There are at least a few real-world examples that have been studied.

There's a strong argument that our rational intelligence evolved to serve the drive to survive--and that emotions are the signals of that drive. It's becoming more clear that even though most animals aren't capable of symbolic reasoning and language, they do experience emotions and make decisions accordingly.

Who is programming a computer to feel emotions? I'm not aware of any significant work in the direction. (If anyone is, please link it!)

Too be fair, labor for those tasks is so cheap to make robots uncompetitive price wise.

The money is in putting that lawyer out of work nor hector, the immigrant strawberry picker

>* Who is programming a computer to feel emotions? I'm not aware of any significant work in the direction. (If anyone is, please link it!)*

Affectiva is working on reading emotions, and there is some philosophy work on whether or not an AI can feel emotions [1]. A Russian group claims they will release an emotional robot around 2018, but scant details on what that even means [2]. More scholarly work [3] [4] [5] is ongoing, but slow going.

[1] https://en.wikipedia.org/wiki/Philosophy_of_artificial_intel...

[2] http://www.dailymail.co.uk/sciencetech/article-3680874/Scien...

[3] http://www.sciencedirect.com/science/article/pii/S0747563204...

[4] http://link.springer.com/chapter/10.1007/978-94-009-2792-6_1...

[5] https://books.google.com/books?hl=en&lr=&id=OqbMnWDKIJ4C&oi=...

I'm not sure what you are responding to, but general-purpose humanoid robots that could be shown how to do things like picking strawberries (rather than a purpose-built strawberry picker) would literally be the world's biggest industry by staggering margins overnight.

The money is in both, and arguably there are greater knock-on effects from automating "labor for those tasks is so cheap to make robots uncompetitive price wise".

A humaniform robot imbued with sentience and what little progress we've made on (emulating?) sapience is sufficient to substantially (not entirely replace) disrupt the human labor demand from the following industries.

  * Residential lawn care for Single Family Residences, a $10B annual industry in the US alone. http://bit.ly/2eu4Ulm
  * Residential maid service, a $30B annual industry in the US alone. http://bit.ly/2eSQ3oL
  * Residential home senior health care, a $180B annual industry in the US alone. http://bit.ly/1ufk95Y
  * If residential meal prep and cooking were mostly or possibly entirely automated, then that can convert the personal chef
    industry from hundreds of millions into one that poaches a significant portion of the $780B fast casual industry.
    http://bit.ly/2dJweO0

That's just four industries, with a combined annual revenue of $1T in the US alone, with tremendous consumer demand. There are lots of other possible industries that can be disrupted with such a "simple" robot, especially in the commercial sectors, and globally. It is not a stretch to say such a robot is a multi-trillion dollar opportunity.

Suppose you offer a $100K humaniform robot with OTA software updates that can perform most household chores, geofence young children into a specific locality, and basically the sense range, kinesthetic sense of a human, object recognition depth of current AI software, but with only the sapience of what we associate with AI today. Amortize the payments over 30 years, and I expect a billion households will beat a path to your door and happily pay that amount.

It would be hard to predict the knock-on economic and other effects of the development of mass-produced personal household robots. Probably similar to trying to predict the demand upon lithium supplies a century from when the Model T was introduced. Perhaps something similarly unexpected like divorce rates dramatically drop due to fewer domestic disputes over household chores because most of those chores disappear.

My thesis is that the people building robots are not dumb, and already realize the financial potential in figuring out sentience, and already are aware at some level that the sentience I speak of is incredibly difficult to build because we understand so little about it, and the AI field focuses upon coming in at the problem from the sapience angle/level for the good reason that sapience is closer to our current form of computer technology than sentience.

I briefly worked on a molecular dynamics project which was for small RNA fragments with atoms numbering in the hundreds. It needed quantum effects simulations to understand two states of the molecule, which could not be explained otherwise. It also required clusters of computers to finish. Modeling proteins is an entire different game. Especially considering the infinite possibilities of proteins and the billions of different proteins that exist in the wild.