(And let me be clear that none of this is an argument that AGI is near. I'm saying that confidence that it is far is unfounded.)

First, there are many cases in science where experts were totally blindsided by breakthroughs. The discovery of controlled fission is probably the most famous example. This shouldn't be surprising - the reason that a breakthrough is a breakthrough is because it adds fundamentally new knowledge. You could only have predicted the breakthrough if you somehow knew that this unknown knowledge was there, waiting to be found. But if you knew that, you'd probably be the one to make the breakthrough in the first place.

Second, most claims about the impossibility of near-term AGI are totally unscientific. By that, I mean that they aren't based on a successful theory of falsifiable predictions. What we'd want, in order to have any confidence, is a theory that can make testable predictions about what will and won't happen in the short term. Then, if those predictions turn out to be true, we can gain confidence in the theory. But this isn't what we get. What we get is people saying "We have no idea how to do x, y, and z, therefore it won't happen in the next 50 years". I don't see any evidence that people were able to predict even the incremental progress we've seen say, two years out. The fact is that when someone says "it'll take 50 years" that's just sort of a gut feeling, and people will almost certainly be making that same prediction the year before it actually happens.

Third, I think people have too narrow a view about what they imagine AGI might look like. People tend to envision something like HAL, that passes Turing tests, can explain chains of reasoning, and has comprehensible motivations. Let's consider the case of abstract reasoning, which is something thought to be very difficult. We tried and failed for decades to build vision systems based on methods of abstract reasoning, e.g. "detect edges, compose edges into shapes, build a set of spatial relationships between those shapes, etc". But humans don't use abstract methods in their visual cortex, they use something a lot more like a DNN. The mistake is in thinking that because the mechanism of successful machine vision resembles human vision, therefore the mechanism of successful machine reasoning must resemble human reasoning. But its quite possible that we'll simply train a DNN by brute force to evaluate causal relationships by "magic", i.e. in a way that doesn't show any evidence of the sort of step-by-step reasoning humans use. You can already see this happening - when a human learns to play breakout, they start by forming an abstract conception of the causal relationships in the game. This allows a human to learn really really fast. But with a DNN, we just brute force it. It never develops what we would consider "understanding" of the game, it just _wins_.

Sorry the third point was so long, let me summarize: We think some things are hard because we don't know how to do them the way that we think humans do them. But that doesn't serve as evidence that there isn't an easy way to do them that is just waiting to be discovered.