You exemplify exactly the attitude he's trying to combat. "Oh, nobody knows anything, let's not care about consequences and do whatever."
Are you sure you read the essay? That's literally the question he answers.
At any rate, we do have more than '0 information', and if you make an honest effort to think of what to do you can likely come up with better than 'random actions' for helping (as many have).
My reading of the article is that he keeps calling for action without specifying what that action should be and trying to justify it by saying he can't know what AGI would look like (so he can't really say what we can do to prevent it).
>> if you make an honest effort to think of what to do you can likely come up with better than 'random actions' for helping (as many have).
Sure. If my research gets up one day and starts self-improving at exponential rates I'll make sure to reach for th
"We don't know how AGI will arise; we don't know when; we don't know why; we don't know anything at all about it and we won't know anything about it until it's too late to do anything anyway; We must act now!!"
The question is- if we don't know anything about this unknowable threat, how can we protect ourselves against it? In fact, since we're starting from 0 information, anything we do has equal chances of backfiring and bringing forth AGI as it has of actually preventing it. Yudkowski is calling for random action, without direction and without reason.
Besides, if Yudkowski is none the wiser about AGI than anyone else, then how is he so sure that AGI _will_ happen, as he insists it will?
Yudkowski is fumbling around in the dark like everyone else in AI. Except he (and a few others) has decided that it's a good strategy, under the circumstances, to raise a hell of a racket. "It's dark!" he yells. "Beware of the darkness!". Yeah OK, friend. It's dark- we can all tell. Why don't you pipe down and let us find the damn light?