Healthcare providers are not allowed to charge publicly-insured patients for covered services. Insurance companies are not allowed to insure for covered services. So for covered services (e.g. lab tests and doctor visits) the only available options is public. For non-covered services (e.g. ambulance, dental, prescription drugs, etc.) you pay out of pocket or use private insurance.

Many healthcare providers themselves are private. For example, doctors are independent contractors, not government employees, and their offices are private businesses. But they get paid for services by the government (why we call if a single-payer system rather than a public system).