It's only close to true for those kids that would have done reasonably anyway.
For those kids that don't have the kind of family environment that will allow them to catch up, it's gonna be a long haul. And they may never get there. That weighs on them for the rest of their time — on the education system, while they're still in it. And it weighs on us, in terms of the resources they take away from other children in education, and then on wider society later in life.
Since the lockdowns, there seems to have been a change in the mindset of some groups — skipping school has been made acceptable. There's a whole bunch of kids that might never return to education as a normal routine. It's likely the hangover will be long and painful for some groups.
Schools aren't the solution to the problem you're describing (children with bad homes) and was a problem before COVID.
Bigger picture is that it is better to keep a larger portion of the population from dying or developing long term complications than slightly (very) protect a few children that are already being abused.
If you're really going to stand by your position then I suggest a reasonable compromise and that is expanding access to licensed professional social workers and community health programs that are there to help parents and children succeed in life.
Kids can catch up. I think people perceive the less than 2 years of remote school that these kids are irreversibly behind in some huge proportion.
The good benefit of not dying easily outweighs the bad effect of a slightly less optimal education for a brief period of time.