This has historically always been an issue in many workplaces. Especially in jobs were people are "interchangeable" and easily replaced and has no leverage, because in those kinds of job the threshold before having the kind of conversation you related becomes much higher, and the incentive for listening lower.
The infamous Triangle Waistcoat Factory fire [1], for example, was as lethal as it was because employers locked the factory doors to prevent unauthorised breaks.
The worst excesses were stopped because they outright killed people and the technological alternatives were not there. But increasingly technology is becoming a way of virtually caging people instead of actually caging them.
[1] https://en.wikipedia.org/wiki/Triangle_Shirtwaist_Factory_fi...
If you do things differently, you open yourself up to criticism and if anything does go bad, you're likely never going to hear the end of it.
Good on you for stopping that vicious circle and being an example of a good manager.
Pickup phone at the start of the day, check in, toss it in a pocket and go to work. Check out, put phone in locker or some storage... done.
But just to expand on that.
I think this is one of those things that also kinda demonstrates a lack of faith and trust between the employer and employee and can be damaging to the whole relationship.
When I had my first job as a manager (low level technical support job) I was a jerk. Not in what I said but in enforcing rules about what is on people's PCs and etc, because that was how that place operated so I did too. One guy on the verge of quitting asked me "Does any of this really matter / help me do my job?" I realized ... probably not / this was a total hassle for me, and him, and everyone. It was just a big distrusting type environment we had going on. I told him and the rest of my team "I'm not checking PCs anymore or anything like that, just be responsible, make good choices."
What happened? The team was happier, and nothing bad happened. I was happier at work, so were they, and I saw customer satisfaction (granted that's a shaky metric) go up ... I assume because everyone was happier / more pleasant to talk to with the customers.
Ages later... I still feel kinda dumb about the whole thing. I really emulated the whole asshole hall monitor type thing for a while. It was completely without value / detrimental.