That's not a meaningful argument, the excuse for having parents place absurd limits on screen time and video games was the same as with smartphones in schools, that they're distracting and addictive, would affect academic performance, and would affect kids in the classroom because they'd be talking about their TV shows and video games or thinking about watching/playing them during class.

It only feels different to you because you have the benefit of hindsight on one of them but not the other.

They are pointing out the more correct analogy. The analogy to banning smartphones in school, would be banning videogames and tv in school, not banning those things altogether. Teachers I've heard speak on the topic, largely seem to want to limit smartphones in class as they have observed student use of smartphones during class to interfere with their teaching and their students learning. This is the same reason why video games are banned in class. Not as you say due to students simply speaking about video games, but because students playing video games during class is distracting for the student playing the game, for other students, and for the teacher.