The nature of unknowns is that you don't know them.

What's the probability of AI singularity? It has never happened before so you have no priors and any number you assign will be pure speculation.

Same is true about anything you're trying to forecast, by definition of it being in the future. And yet people have figured out how to make predictions more narrow than shrugging.

"And the general absolutist tone of the community. The people involved all seem very... Full of themselves ?"

>And yet people have figured out how to make predictions more narrow than shrugging

And?

>> It has never happened before

> Same is true about anything you're trying to forecast, by definition of it being in the future

There might be some flaws in this line of reasoning...

Yes, but making it more nuanced still doesn't change the point.

Singularity obviously never happened before, and if anyone bothered to read up on what they're talking about, they'd realize that no one is trying to predict what happens then, because the singularity is defined as the time when changes accelerate to such a degree that we have no baseline to make any predictions whatsoever.

So when people speculate on when that is, they're trying to forecast the point forecasting breaks; they do it by extrapolating from known examples and trends, to which we do have baselines.

Or, in short: we know how it is to ride an exponent, we just never rode one long enough to fall off of it; predicting singularity is predicting when the exponent gets steep enough we can't follow, which is not unlike predicting any other trend people do. Same methods and caveats apply.

"It is difficult to make predictions, especially about the future."

Most of the time we make predictions based on how similar events happened in the past. For completely novel situations it's close to impossible to make a prediction and reckless to base policy on such a prediction.

That's strictly true, but I feel like you're misunderstanding something. Most people aren't actually doing anything truly novel, hence very few people ever actually have to even attempt to predict things in this way.

But it was necessary at the beginning of flight and the flight to the moon would've never been possible either without a few talented people being able to make predictions about scenarios they knew little about.

There are just way too many people around nowadays, which is why most of us never get confronted with such novel topics and consequently we don't know how to reason about it

> They even know how to put bounds on the unknowns and their own lack of information.

No they don't. They think they can do this because they've accidentally reinvented the philosophy "logical positivism", which philosophers gave up on because it doesn't work. (This is similar to how they accidentally reinvented reconstructing arguments and called it "steelmanning".)

https://metarationality.com/probability-limitations