Yes, which is what I think AI contributes to
Especially with reckless ideas like "vibe coding" going around
"Vibe coding" is primarily being done by people who already weren't learning to program. If programming was taught as ubiquitously and early as arithmetic, they either wouldn't be vibe coding or they were never going to program in the first place because for whatever reason they were unable or unwilling to develop the skill. So you could see this as a loss of potential, but I suspect it is far more of a democratization of programming.
If we're looking at parallels, we should be looking at WYSIWYG webpage creation tools rather than calculators. Did that destroy knowledge of HTML and CSS?
Now apply that to LLM used to writing, research, reading comprehension, summarizing, debating, and so on, in the same school system
That's the point of learning arithmetic first without calculators.
The question is whether a learned skill is “much worse” in the presence of a tool. You seem to be agreeing that it isn’t (or at least doesn’t have to be), but that danger lies in just never learning the skill.