It was only because of all the controversy over migrant workers rights during the prelude to FIFA World Cup in Qatar that holding your employees passports became illegal.

Before that, employers had the right to do so and it was (might still be) the norm.

Refusing to hand over your passport is like refusing to hand over your passport at immigration.