I don't know if AGI is possible or not but I do think this is a valid question. The AI risk people gloss over this fundamental question and proceed directly to the question of "AI safety". This is a major weakness in their argument.