Restricting religion and regulating a factory are pretty different. The first runs directly afoul of the constitution, while the second is clearly within the purview of regulators and public health officials.

I was thinking of Jacobson v Massachusetts in my original comment, which upheld state and local governments' broad authority in public health.

Here's a balanced view of the powers affirmed in Jacobson and how that has changed over time to better protect individual liberty: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1449224/

Note however that applies to individuals, not factories or places of business which are subject to much more regulation.