The problem is if there's just one teacher tutoring, one quiz master who points to sources it controls or simply invents as its "proof". My concern is that AI will be trusted more than a single book or a single teacher, precisely because it claims to neutrally encompass a consensus already obtained through all those sources. [It's a tertiary source masquerading as one offering primary and secondary sources, without the ability to differentiate or trust one source more than another].
This is exactly why the frequent comparison to calculators is so misguided: Because calculators are always right. To whatever degree LLM proponents believe LLMs are simply useful tools like calculators, that will be the same degree to which they fail to check for bullshit.
If those things are a problem, we have already failed. Because if your current education system is built upon taking things at face value, the harm is not in the shoveled knowledge but in building a student who accepts what is presented as unquestionable fact.
My child is an phenomenal bullshit discriminator. So much so that they have been correctly correcting all of their teachers in every grade they have been in. The real education is learning how to use the tools we have available to learn anything you want. And in that way, the LLM is most powerful thing we have since the printed book.