What worries me are the impacts of constant surveillance that we can't see. What does it mean when every child gets assigned into various buckets like "smart/stupid", "constantly late", "impulsive", "has few fiends", etc by companies who are constantly tracking their grades and test scores in every subject as they age, along with their location and social networks and online habits. What happens when that data ends up with data brokers and sold to everyone from potential employers, to governments, to insurance companies, to criminals and scammers, and to AI companies for use as training data.

We're already seeing things like predictive policing, surveillance pricing, judges using AI to write rulings and parole-boards using AI to decide the fate of prisoners, extensive (and sometimes illegal) background checks used by landlords and employers. AI is making it ever easier to trawl through the massive piles of data being collected and output results with random errors people will probably never be told about even when they're used to make decisions that have major impacts on their lives.

It's become so normalized, even by their own parents, that kids growing up in this environment don't even question it.