If I take what your first sentence as true then it follows that no one (LeCun, Ng, Bostrom) is an expert, nor do any of them exceed the corner-store psychic's ability to predict what will come of AI.

I don't find that too hard to believe, personally, but it's unfair to single out Bostrom as a charlatan if you really believe that there's no way of knowing what will happen post-development of human-level AI.

I'm singling out Bostrom because he attempts to predict events over a much larger timescale, which diminishes the value of those predictions, because they are contingent on so many other developments we can't know. LeCun and Ng, as I said, are much more modest in their forecasts, aiming at the nearer term. Bostrom is basically dealing in the outcomes of very long joint probabilities, which, multiplied by the probability of each individual variable, is very low.

But LeCun and Ng also made long term predictions. Lecun said it's unlikely that AI will be dangerous, and Ng said it's unlikely to happen in our lifetime. Both of those predictions are just as speculative as Bostrom's.

>Bostrom is basically dealing in the outcomes of very long joint probabilities, which, multiplied by the probability of each individual variable, is very low.

That's not how probability works. You can't just assume Bostrom is wrong by default. That might work for futurists like Hanson or Kurzweil, which do that. But Bostrom isn't making a huge number of assumptions. His book presents a large number of possible futures, and the dangers of each.

Bostrom has to stack assumption on tenuous assumption because that's the nature of the problem. but some things get more certain the closer to a genuine superintelligence one gets, such as the assertion that it is the last invention humans need devise.

Bostrom is also using the tools of human philosophy assuming they are general enough to apply to superintelligence. So he comes off as inherently anthropomorphizing even as he warns against doing just that.

He said Superintelligence was a very difficult book to write and that's probably part of what he meant by "difficult."

There is plenty to doubt. One big doubt about the danger of AI is that AI is not an animal. It is not alive like an animal. It has no death like an animal has. It doesn't need a "self." It doesn't propagate genes. It did not evolve through natural selection. So, except for Bostrom's use of whole brain emulation as a yardstick, there isn't much of the commonplace things that makes humans dangerous that needs to be in an AI.

But if the ideas of "strategic advantage" are in general correct, in the way Bostrom uses them, then Bostrom is right to say we are like a child playing with a bomb.

To quote Wittgenstein: “Whereof one cannot speak, thereof one must be silent.” Writing about the the unwritable makes books hard to write. The tools of philosophy, in the 20th century at least, rely heavily on obscurantism and impenetrable jargon, to which Bostrom is no stranger.

I found Superintelligence very readable. Very little jargon. I suspect he is wrong and that superintelligence is the Y2K of the 21st century. I suspect that super-machine-brains will dominate us as much as power plants sneer at our puny energy output. The most alarming scenarios assume anything, like practical nanotech, could be created by bootstrapping from a powerful brain in a box. And once you make that assumption, kaboom.

Just don't enhance the chimps. That would be bad.

Joint probability goes both ways: Ng's prediction that X% of workers will become obsolete is just as unlikely as any future, including Bostroms, as any long string of probable outcomes is highly unlikely.

We can't know exactly what will happen, but we can make informed speculation. Bostrom studied many different scenarios for the future of AI, and the arguments for and against them. He's even contributed to AI safety research, the study of methods to control future intelligent algorithms.

So yes he absolutely is more informed than a psychic, or even an expert in today's machine learning. And he isn't even making very specific predictions. He's not giving an exact timetable of when we will have AI, or what algorithms it will use. Just a very general prediction that we will get superintelligence someday, and that it will be very dangerous if we don't solve the control problem.

These seem like very reasonable statements, and have strong arguments for them IMO.