I think you're right that it's possible in principle for a Web server to enforce use of DoD CAC (enforcing the client cert being in the DoD PKI) without itself using a DoD PKI cert on the server side.

That said there's little benefit to it, users who haven't jumped through hoops to install DoD root CA certs won't typically be able to get their browsers to present them to the remote server in the first place, and if we're willing to jump through those hoops then there's no good reason for the DoD server not to have a DoD PKI cert.

FWIW I just tested it and yes you can run a web server using a commercial server cert that enforces client PKI tied to the client having a DoD PKI cert. It works just fine.

It’s been a while, but I’ve used file-backed client certs issued by a private CA in an ordinary browser without installing anything into the trust store, and it worked fine. I don’t see why a client cert using PKCS11 or any other store would work any differently. Why would the browser want to verify a certificate chain at all?

I guess you could flag the leaf (user) cert as ultimately trusted and that should be fine, but if the browser doesn't see that trust notation, and does see an intermediate CA, it's going to try to pull that back to a trusted root.

One way or the other the user will have to fiddle with browser settings to make a CAC work, either to tell the browser to trust their cert explicitly, or to have the browser trust DoD certs.

Critically you probably want to use a custom CA for client certs. The usual implementation logic in servers is "is this cert from the client signed by one I trust?". If that CA is LetsEncrypt say then that's a lot of certificates that will pass that check.